draft-ietf-nfsv4-rfc3530-migration-update-08.txt   rfc7931.txt 
NFSv4 D. Noveck, Ed. Internet Engineering Task Force (IETF) D. Noveck, Ed.
Internet-Draft HPE Request for Comments: 7931 HPE
Updates: 7530 (if approved) P. Shivam Updates: 7530 P. Shivam
Intended status: Standards Track C. Lever Category: Standards Track C. Lever
Expires: August 4, 2016 B. Baker ISSN: 2070-1721 B. Baker
ORACLE ORACLE
February 1, 2016 July 2016
NFSv4.0 migration: Specification Update NFSv4.0 Migration: Specification Update
draft-ietf-nfsv4-rfc3530-migration-update-08
Abstract Abstract
The migration feature of NFSv4 allows for responsibility for a single The migration feature of NFSv4 allows the transfer of responsibility
file system to move from one server to another, without disruption to for a single file system from one server to another without
clients. Recent implementation experience has shown problems in the disruption to clients. Recent implementation experience has shown
existing specification for this feature in NFSv4.0. This document problems in the existing specification for this feature in NFSv4.0.
identifies the problem areas and provides revised specification text This document identifies the problem areas and provides revised
which updates the NFSv4.0 specification in RFC7530. specification text that updates the NFSv4.0 specification in RFC
7530.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This is an Internet Standards Track document.
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 7841.
This Internet-Draft will expire on August 4, 2016. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7931.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 20 skipping to change at page 3, line 16
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3
3.2. Data Type Definitions . . . . . . . . . . . . . . . . . . 5 3.2. Data Type Definitions . . . . . . . . . . . . . . . . . . 5
4. Background . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Background . . . . . . . . . . . . . . . . . . . . . . . . . 5
5. Client Identity Definition . . . . . . . . . . . . . . . . . 7 5. Client Identity Definition . . . . . . . . . . . . . . . . . 7
5.1. Differences from Replaced Sections . . . . . . . . . . . 7 5.1. Differences from Replaced Sections . . . . . . . . . . . 7
5.2. Client Identity Data Items . . . . . . . . . . . . . . . 8 5.2. Client Identity Data Items . . . . . . . . . . . . . . . 8
5.2.1. Client Identity Structure . . . . . . . . . . . . . . 8 5.2.1. Client Identity Structure . . . . . . . . . . . . . . 9
5.2.2. Client Identity Shorthand . . . . . . . . . . . . . . 10 5.2.2. Client Identity Shorthand . . . . . . . . . . . . . . 11
5.3. Server Release of Client ID . . . . . . . . . . . . . . . 13 5.3. Server Release of Client ID . . . . . . . . . . . . . . . 13
5.4. Client Id String Approaches . . . . . . . . . . . . . . . 13 5.4. Client ID String Approaches . . . . . . . . . . . . . . . 14
5.5. Non-Uniform Client Id String Approach . . . . . . . . . . 15 5.5. Non-uniform Client ID String Approach . . . . . . . . . . 16
5.6. Uniform Client Id String Approach . . . . . . . . . . . . 16 5.6. Uniform Client ID String Approach . . . . . . . . . . . . 16
5.7. Mixing Client Id String Approaches . . . . . . . . . . . 18 5.7. Mixing Client ID String Approaches . . . . . . . . . . . 18
5.8. Trunking Determination when Using Uniform Client Id 5.8. Trunking Determination when Using Uniform Client ID
Strings . . . . . . . . . . . . . . . . . . . . . . . . . 19 Strings . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.9. Client Id String Construction Details . . . . . . . . . . 26 5.9. Client ID String Construction Details . . . . . . . . . . 26
6. Locking and Multi-Server Namespace . . . . . . . . . . . . . 27 6. Locking and Multi-Server Namespace . . . . . . . . . . . . . 28
6.1. Lock State and File System Transitions . . . . . . . . . 28 6.1. Lock State and File System Transitions . . . . . . . . . 28
6.1.1. Migration and State . . . . . . . . . . . . . . . . . 28 6.1.1. Migration and State . . . . . . . . . . . . . . . . . 29
6.1.1.1. Migration and Client IDs . . . . . . . . . . . . 30 6.1.1.1. Migration and Client IDs . . . . . . . . . . . . 31
6.1.1.2. Migration and State Owner Information . . . . . . 31 6.1.1.2. Migration and State Owner Information . . . . . . 32
6.1.2. Replication and State . . . . . . . . . . . . . . . . 35 6.1.2. Replication and State . . . . . . . . . . . . . . . . 36
6.1.3. Notification of Migrated Lease . . . . . . . . . . . 35 6.1.3. Notification of Migrated Lease . . . . . . . . . . . 36
6.1.4. Migration and the Lease_time Attribute . . . . . . . 38 6.1.4. Migration and the lease_time Attribute . . . . . . . 39
7. Server Implementation Considerations . . . . . . . . . . . . 39 7. Server Implementation Considerations . . . . . . . . . . . . 39
7.1. Relation of Locking State Transfer to Other Aspects of 7.1. Relation of Locking State Transfer to Other Aspects of
File System Motion . . . . . . . . . . . . . . . . . . . 39 File System Motion . . . . . . . . . . . . . . . . . . . 39
7.2. Preventing Locking State Modification During Transfer . . 40 7.2. Preventing Locking State Modification during Transfer . . 41
8. Additional Changes . . . . . . . . . . . . . . . . . . . . . 44 8. Additional Changes . . . . . . . . . . . . . . . . . . . . . 44
8.1. Summary of Additional Changes from Previous Documents . . 44 8.1. Summary of Additional Changes from Previous Documents . . 45
8.2. NFS4ERR_CLID_INUSE definition . . . . . . . . . . . . . . 45 8.2. NFS4ERR_CLID_INUSE Definition . . . . . . . . . . . . . . 45
8.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER . . . . . . . 45 8.3. NFS4ERR_DELAY Return from RELEASE_LOCKOWNER . . . . . . . 45
8.4. Operation 35: SETCLIENTID - Negotiate Client ID . . . . . 46 8.4. Operation 35: SETCLIENTID -- Negotiate Client ID . . . . 46
8.5. Security Considerations for Inter-server Information 8.5. Security Considerations for Inter-server Information
Transfer . . . . . . . . . . . . . . . . . . . . . . . . 50 Transfer . . . . . . . . . . . . . . . . . . . . . . . . 51
8.6. Security Considerations Revision . . . . . . . . . . . . 50 8.6. Security Considerations Revision . . . . . . . . . . . . 51
9. Security Considerations . . . . . . . . . . . . . . . . . . . 51 9. Security Considerations . . . . . . . . . . . . . . . . . . . 52
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 51 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 52
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 51 10.1. Normative References . . . . . . . . . . . . . . . . . . 52
11.1. Normative References . . . . . . . . . . . . . . . . . . 51 10.2. Informative References . . . . . . . . . . . . . . . . . 52
11.2. Informative References . . . . . . . . . . . . . . . . . 51 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 53
Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 52 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 54
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 52
1. Introduction 1. Introduction
This document is a standards track document which corrects the This Standards Track document corrects the existing definitive
existing definitive specification of the NFSv4.0 protocol, in specification of the NFSv4.0 protocol described in [RFC7530]. Given
[RFC7530]. Given this fact, one should take the current document this fact, one should take the current document into account when
into account when learning about NFSv4.0, particularly if one is learning about NFSv4.0, particularly if one is concerned with issues
concerned with issues that relate to: that relate to:
o File system migration, particularly when it involves transparent o File system migration, particularly when it involves transparent
state migration. state migration.
o The construction and interpretation of the nfs_client_id4 o The construction and interpretation of the nfs_client_id4
structure and particularly the requirements on the id string structure and particularly the requirements on the id string
within it, referred to below as a "client id string". within it, referred to below as a "client ID string".
2. Conventions 2. Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
3. Definitions 3. Definitions
3.1. Terminology 3.1. Terminology
The following definitions are provided for the purpose of providing The following definitions are included to provide an appropriate
an appropriate context for the reader. This section is derived from context for the reader. This section is derived from Section 1.5 of
Section 1.5 of [RFC7530] but has been adapted to the needs of this [RFC7530] but has been adapted to the needs of this document.
document.
Boot instance id: A boot instance id, is an identifier, such as a Boot Instance Id: A boot instance id is an identifier, such as a
boot time, allowing two different instances of the same client to boot time, allowing two different instances of the same client to
be reliably distinguished. A boot instance id is opaque to the be reliably distinguished. A boot instance id is opaque to the
server and is often used as the verifier field in the server and is often used as the verifier field in the
nfs_client_id4 structure which identifies the client to the nfs_client_id4 structure, which identifies the client to the
server. server.
Client: A client is an entity that accesses the NFS server's Client: A client is an entity that accesses the NFS server's
resources. The client may be an application that contains the resources. The client may be an application that contains the
logic to access the NFS server directly. The client may also be logic to access the NFS server directly. The client may also be
the traditional operating system client that provides remote file the traditional operating system client that provides remote file
system services for a set of applications. system services for a set of applications.
With reference to byte-range locking, the client is also the With reference to byte-range locking, the client is also the
entity that maintains a set of locks on behalf of one or more entity that maintains a set of locks on behalf of one or more
applications. This client is responsible for crash or failure applications. This client is responsible for crash or failure
recovery for those locks it manages. recovery for those locks it manages.
Note that multiple clients may share the same transport and Note that multiple clients may share the same transport and
connection, and multiple clients may exist on the same network connection, and multiple clients may exist on the same network
node. node.
Client ID: A client ID is a 64-bit quantity (in the form of a Client ID: A client ID is a 64-bit quantity (in the form of a
clientid4) used as a unique, shorthand reference to particular clientid4) used as a unique, shorthand reference to a particular
client instance, identified by client-supplied verifier (in the client instance, identified by a client-supplied verifier (in the
form of a boot instance id) and client id string. The server is form of a boot instance id) and client ID string. The server is
responsible for supplying the client ID. responsible for supplying the client ID.
File System: A file system is the collection of objects on a server File System: A file system is the collection of objects on a server
that share the same fsid attribute (see Section 5.8.1.9 of that share the same fsid attribute (see Section 5.8.1.9 of
[RFC7530]). [RFC7530]).
Grace period: A grace period is an interval of time during which the Grace Period: A grace period is an interval of time during which the
server will only grant locking requests to reclaim existing locks server will only grant locking requests to reclaim existing locks
but not those which create new locks. This give clients an but not those that create new locks. This gives clients an
opportunity to re-establish locking state in response to a opportunity to re-establish locking state in response to a
potentially disruptive event. The grace period may be general, to potentially disruptive event. The grace period may be general to
help deal with server reboot, or file-system-specific, to deal help deal with server reboot, or it may be specific to a file
with file system migration when transparent state migration is not system to deal with file system migration when transparent state
provided. migration is not provided.
Lease: A lease is an interval of time defined by the server for Lease: A lease is an interval of time defined by the server for
which the client is irrevocably granted a lock. At the end of a which the client is irrevocably granted a lock. At the end of a
lease period the lock may be revoked if the lease has not been lease period, the lock may be revoked if the lease has not been
extended. The lock must be revoked if a conflicting lock has been extended. The lock must be revoked if a conflicting lock has been
granted after the lease interval. granted after the lease interval.
All leases granted by a server have the same fixed duration. Note All leases granted by a server have the same fixed duration. Note
that the fixed interval duration was chosen to alleviate the that the fixed interval duration was chosen to alleviate the
expense a server would have in maintaining state about variable- expense a server would have in maintaining state about variable-
length leases across server failures. length leases across server failures.
Lock: The term "lock" is used to refer to record (byte-range) locks Lock: The term "lock" is used to refer to record (byte-range) locks
as well as share reservations unless specifically stated as well as share reservations unless specifically stated
skipping to change at page 5, line 22 skipping to change at page 6, line 17
Stateid: A stateid is a 128-bit quantity returned by a server that Stateid: A stateid is a 128-bit quantity returned by a server that
uniquely identifies the open and locking states provided by the uniquely identifies the open and locking states provided by the
server for a specific open-owner or lock-owner/open-owner pair for server for a specific open-owner or lock-owner/open-owner pair for
a specific file and type of lock. a specific file and type of lock.
Trunking: A situation in which multiple physical addresses are Trunking: A situation in which multiple physical addresses are
connected to the same logical server. connected to the same logical server.
Verifier: A verifier is a quantity, in the form of a verifier4, that Verifier: A verifier is a quantity, in the form of a verifier4, that
allows one party to an interaction to be aware of a re- allows one party to an interaction to be aware of a
initialization or other significant change to the state of the reinitialization or other significant change to the state of the
other party. In [RFC7530], this term most often designates the other party. In [RFC7530], this term most often designates the
verifier field of an nfs_client_id4, in which a boot instance id verifier field of an nfs_client_id4, in which a boot instance id
is placed to allow the server to determine when there has been a is placed to allow the server to determine when there has been a
client reboot, making it necessary to eliminate locking state client reboot, making it necessary to eliminate locking state
associated with the previous instance of the same client. associated with the previous instance of the same client.
3.2. Data Type Definitions 3.2. Data Type Definitions
This section contains a table which shows where data types referred This section contains a table that shows where data types referred to
to in this document are defined. in this document are defined.
+-----------------+------------------------------+ +-----------------+--------------------------------+
| Item | Section | | Item | Section |
+-----------------+------------------------------+ +-----------------+--------------------------------+
| cb_client4 | Section 2.2.11 in [RFC7530] | | cb_client4 | Section 2.2.11 in [RFC7530] |
| clientaddr4 | Section 2.2.10 in [RFC7530] | | clientaddr4 | Section 2.2.10 in [RFC7530] |
| clientid4 | Section 2.1 in [RFC7530] | | clientid4 | Section 2.1 in [RFC7530] |
| lock_owner4 | Section 2.2.14 in [RFC7530] | | lock_owner4 | Section 2.2.14 in [RFC7530] |
| nfs_client_id4 | Section 5.2.1 | | nfs_client_id4 | Section 5.2.1 (this document) |
| open_owner4 | Section 2.2.13 in [RFC7530] | | open_owner4 | Section 2.2.13 in [RFC7530] |
| verifier4 | Section 2.1 in [RFC7530] | | verifier4 | Section 2.1 in [RFC7530] |
+-----------------+------------------------------+ +-----------------+--------------------------------+
4. Background 4. Background
Implementation experience with transparent state migration has Implementation experience with transparent state migration has
exposed a number of problems with the then-existing specifications of exposed a number of problems with the then existing specifications of
this feature, in [RFC7530] and predecessors. The symptoms were: this feature in [RFC7530] and predecessors. The symptoms were:
o After migration of a file system, a reboot of the associated o After migration of a file system, a reboot of the associated
client was not appropriately dealt with, in that the state client was not appropriately dealt with, in that the state
associated with the rebooting client was not promptly freed. associated with the rebooting client was not promptly freed.
o Situations can arise whereby a given server has multiple leases o Situations can arise whereby a given server has multiple leases
with the same nfs_client_id4 (consisting of id and verifier with the same nfs_client_id4 (consisting of id and verifier
fields), when the protocol clearly assumes there can be only one. fields), when the protocol clearly assumes there can be only one.
o Excessive client implementation complexity since clients have to o Excessive client implementation complexity since clients have to
deal with situations in which a single client can wind up with its deal with situations in which a single client can wind up with its
locking state with a given server divided among multiple leases locking state with a given server divided among multiple leases
each with its own clientid4. each with its own clientid4.
An analysis of these symptoms leads to the conclusion that existing An analysis of these symptoms leads to the conclusion that existing
specifications have erred. They assume that locking state, including specifications have erred. They assume that locking state, including
both state ids and clientid4's, should be transferred as part of both state ids and clientid4s, should be transferred as part of
transparent state migration. The troubling symptoms arise from the transparent state migration. The troubling symptoms arise from the
failure to describe how migrating state is to be integrated with failure to describe how migrating state is to be integrated with
existing client definition structures on the destination server. existing client definition structures on the destination server.
The need for the server to appropriately merge stateids associated The need for the server to appropriately merge stateids associated
with a common client boot instance encounters a difficult problem. with a common client boot instance encounters a difficult problem.
The issue is that the common client practice with regard to the The issue is that the common client practice with regard to the
presentation of unique strings specifying client identity makes it presentation of unique strings specifying client identity makes it
essentially impossible for the client to determine whether or not two essentially impossible for the client to determine whether or not two
stateids, originally generated on different servers are referable to stateids, originally generated on different servers, are referable to
the same client. This practice is allowed and endorsed by the the same client. This practice is allowed and endorsed by the
existing NFSv4.0 specification ([RFC7530]). existing NFSv4.0 specification [RFC7530].
However, upon prototyping of clients implementing an alternative However, upon the prototyping of clients implementing an alternative
approach, it has been found that there exist servers which do not approach, it has been found that there exist servers that do not work
work well with these new clients. It appears that current well with these new clients. It appears that current circumstances,
circumstances, in which a particular client implementation pattern in which a particular client implementation pattern had been adopted
had been adopted universally, has resulted in some servers not being universally, have resulted in some servers not being able to
able to interoperate against alternate client implementation interoperate against alternate client implementation patterns. As a
patterns. As a result, we have a situation which requires careful result, we have a situation that requires careful attention to
attention to compatibility issues to untangle. untangling compatibility issues.
This document updates the existing NFSv4.0 specification ([RFC7530]) This document updates the existing NFSv4.0 specification [RFC7530] as
as follows: follows:
o It makes clear that NFSv4.0 supports multiple approaches to the o It makes clear that NFSv4.0 supports multiple approaches to the
construction of client id strings, including that formerly construction of client ID strings, including those formerly
endorsed by existing NFSV4.0 specifications, and currently widely endorsed by existing NFSV4.0 specifications and those currently
deployed. being widely deployed.
o It explains how clients can effectively use client id strings that o It explains how clients can effectively use client ID strings that
are presented to multiple servers. are presented to multiple servers.
o It addresses the potential compatibility issues that might arise o It addresses the potential compatibility issues that might arise
for clients adopting a previously non-favored client id string for clients adopting a previously non-favored client ID string
construction approach including the existence of servers which construction approach including the existence of servers that have
have problems with the new approach. problems with the new approach.
o It gives some guidance regarding the factors that might govern o It gives some guidance regarding the factors that might govern
clients' choice of a client id string construction approach and clients' choice of a client ID string construction approach and
recommends that clients construct client id strings in manner that recommends that clients construct client ID strings in a manner
supports lease merger if they intend to support transparent state that supports lease merger if they intend to support transparent
migration. state migration.
o It specifies how state is to be transparently migrated, including o It specifies how state is to be transparently migrated, including
defining how state that arrives at a new server as part of defining how state that arrives at a new server as part of
migration is to be merged into existing leases for clients migration is to be merged into existing leases for clients
connected to the target server. connected to the target server.
o It makes further clarifications and corrections to address cases o It makes further clarifications and corrections to address cases
where the specification text does not take proper account of the where the specification text does not take proper account of the
issues raised by state migration or where it has been found that issues raised by state migration or where it has been found that
the existing text is insufficiently clear. This includes a the existing text is insufficiently clear. This includes a
revised definition of the SETCLIENTID operation in Section 8.4 revised definition of the SETCLIENTID operation in Section 8.4,
which replaces Section 16.33 in [RFC7530] which replaces Section 16.33 in [RFC7530].
For a more complete explanation of the choices made in addressing For a more complete explanation of the choices made in addressing
these issues, see [info-migr]). these issues, see [INFO-MIGR].
5. Client Identity Definition 5. Client Identity Definition
This chapter is a replacement for sections 9.1.1 and 9.1.2 in This section is a replacement for Sections 9.1.1 and 9.1.2 in
[RFC7530]. The replaced sections are named "Client ID" and "Server [RFC7530]. The replaced sections are named "Client ID" and "Server
Release of Client ID". Release of Client ID", respectively.
It supersedes the replaced sections. It supersedes the replaced sections.
5.1. Differences from Replaced Sections 5.1. Differences from Replaced Sections
Because of the need for greater attention to and careful description Because of the need for greater attention to and careful description
of this area, this chapter is much larger than the sections it of this area, this section is much larger than the sections it
replaces. The principal changes/additions made by this chapter are: replaces. The principal changes/additions made by this section are:
o It corrects inconsistencies regarding the possible role or non- o It corrects inconsistencies regarding the possible role or non-
role of the client IP address in construction of client id role of the client IP address in construction of client ID
strings. strings.
o It clearly addresses the need to maintain a non-volatile record o It clearly addresses the need to maintain a non-volatile record
across reboots of client id strings or any changeable values that across reboots of client ID strings or any changeable values that
are used in their construction. are used in their construction.
o It provides a more complete description of circumstances leading o It provides a more complete description of circumstances leading
to clientid4 invalidity and the appropriate recovery actions. to clientid4 invalidity and the appropriate recovery actions.
o It presents, as valid alternatives, two approaches to client id o It presents, as valid alternatives, two approaches to client ID
string construction (named "uniform" and "non-uniform") and gives string construction (named "uniform" and "non-uniform") and gives
some implementation guidance to help implementers choose one or some implementation guidance to help implementers choose one or
the other of these. the other of these.
o It adds a discussion of issues involved for clients in interacting o It adds a discussion of issues involved for clients in interacting
with servers whose behavior is not consistent with use of uniform with servers whose behavior is not consistent with use of uniform
client id strings client ID strings.
o It adds a description of how server behavior might be used by the o It adds a description of how server behavior might be used by the
client to determine when multiple server IP addresses correspond client to determine when multiple server IP addresses correspond
to the same server. to the same server.
5.2. Client Identity Data Items 5.2. Client Identity Data Items
The NFSv4 protocol contains a number of protocol entities to identify The NFSv4 protocol contains a number of protocol entities to identify
clients and client-based entities, for locking-related purposes: clients and client-based entities for locking-related purposes:
o The nfs_client_id4 structure which uniquely identifies a specific o The nfs_client_id4 structure, which uniquely identifies a specific
client boot instance. That identification is presented to the client boot instance. That identification is presented to the
server by doing a SETCLIENTID operation. The SETCLIENTID server by doing a SETCLIENTID operation. The SETCLIENTID
operation is described in Section 8.4 which modifies a description operation is described in Section 8.4, which modifies a
in Section 16.33 of [RFC7530] description in Section 16.33 of [RFC7530].
o The clientid4 which is returned by the server upon completion of a o The clientid4, which is returned by the server upon completion of
successful SETCLIENTID operation. This id is used by the client a successful SETCLIENTID operation. This id is used by the client
to identify itself when doing subsequent locking-related to identify itself when doing subsequent locking-related
operations. A clientid4 is associated with a particular lease operations. A clientid4 is associated with a particular lease
whereby a client instance holds state on a server instance and may whereby a client instance holds state on a server instance and may
become invalid due to client reboot, server reboot, or other become invalid due to client reboot, server reboot, or other
circumstances. circumstances.
o Opaque arrays which are used together with the clientid4 to o Opaque arrays, which are used together with the clientid4 to
designate within-client entities (e.g., processes) as the owners designate within-client entities (e.g., processes) as the owners
of opens (open-owners) and owners of byte-range locks (lock- of opens (open-owners) and owners of byte-range locks (lock-
owners). owners).
5.2.1. Client Identity Structure 5.2.1. Client Identity Structure
The basis of the client identification infrastructure is encapsulated The basis of the client identification infrastructure is encapsulated
in the following data structure, which also appears in Section 9.1.1 in the following data structure, which also appears in Section 9.1.1
of [RFC7530]: of [RFC7530]:
struct nfs_client_id4 { struct nfs_client_id4 {
verifier4 verifier; verifier4 verifier;
opaque id<NFS4_OPAQUE_LIMIT>; opaque id<NFS4_OPAQUE_LIMIT>;
}; };
The nfs_client_id4 structure uniquely defines a particular client The nfs_client_id4 structure uniquely defines a particular client
boot instance as follows: boot instance as follows:
o The id field is a variable-length string which uniquely identifies o The id field is a variable-length string that uniquely identifies
a specific client. Although, it is described here as a string and a specific client. Although it is described here as a string and
it is often referred to as a "client string," it should be is often referred to as a "client string", it should be understood
understood that the protocol defines this as opaque data. In that the protocol defines this as opaque data. In particular,
particular, those receiving such an id should not assume that it those receiving such an id should not assume that it will be in
will be in the UTF-8 encoding. Servers MUST NOT reject an the UTF-8 encoding. Servers MUST NOT reject an nfs_client_id4
nfs_client_id4 simply because the id string does not follow the simply because the id string does not follow the rules of UTF-8
rules of UTF-8 encoding. encoding.
The encoding and decoding processes for this field (e.g., use of The encoding and decoding processes for this field (e.g., use of
network byte order) need to result in the same internal network byte order) need to result in the same internal
representation whatever the endianness of the originating and representation whatever the endianness of the originating and
receiving machines. receiving machines.
o The verifier field contains a client boot instance identifier that o The verifier field contains a client boot instance identifier that
is used by the server to detect client reboots. Only if the boot is used by the server to detect client reboots. Only if the boot
instance is different from that which the server has previously instance is different from that which the server has previously
recorded in connection with the client (as identified by the id recorded in connection with the client (as identified by the id
skipping to change at page 10, line 11 skipping to change at page 11, line 23
o The string should be selected so that subsequent incarnations o The string should be selected so that subsequent incarnations
(e.g., reboots) of the same client cause the client to present the (e.g., reboots) of the same client cause the client to present the
same string. The implementer is cautioned against an approach same string. The implementer is cautioned against an approach
that requires the string to be recorded in a local file because that requires the string to be recorded in a local file because
this precludes the use of the implementation in an environment this precludes the use of the implementation in an environment
where there is no local disk and all file access is from an NFSv4 where there is no local disk and all file access is from an NFSv4
server. server.
o The string MAY be different for each server network address that o The string MAY be different for each server network address that
the client accesses, rather than common to all server network the client accesses rather than common to all server network
addresses. addresses.
The considerations that might influence a client to use different The considerations that might influence a client to use different
strings for different network server addresses are explained in strings for different network server addresses are explained in
Section 5.4. Section 5.4.
o The algorithm for generating the string should not assume that the o The algorithm for generating the string should not assume that the
clients' network addresses will remain the same for any set period clients' network addresses will remain the same for any set period
of time. Changes might occur between client incarnations and even of time. Even while the client is still running in its current
while the client is still running in its current incarnation. incarnation, changes might occur between client incarnations.
Changes to the client id string due to network address changes Changes to the client ID string due to network address changes
would result in successive SETCLIENTID operations for the same would result in successive SETCLIENTID operations for the same
client appearing as from different clients, interfering with the client appearing as from different clients, interfering with the
use of the nfs_client_id4 verifier field to cancel state use of the nfs_client_id4 verifier field to cancel state
associated with previous boot instances of the same client. associated with previous boot instances of the same client.
The difficulty is more severe if the client address is the only The difficulty is more severe if the client address is the only
client-based information in the client id string. In such a case, client-based information in the client ID string. In such a case,
there is a real risk that, after the client gives up the network there is a real risk that after the client gives up the network
address, another client, using the same algorithm, would generate address, another client, using the same algorithm, would generate
a conflicting id string. This would be likely to cause an a conflicting id string. This would be likely to cause an
inappropriate loss of locking state. See Section 5.9 for detailed inappropriate loss of locking state. See Section 5.9 for detailed
guidance regarding client id string construction. guidance regarding client ID string construction.
5.2.2. Client Identity Shorthand 5.2.2. Client Identity Shorthand
Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has successfully Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has successfully
completed, the client uses the shorthand client identifier, of type completed, the client uses the shorthand client identifier, of type
clientid4, instead of the longer and less compact nfs_client_id4 clientid4, instead of the longer and less compact nfs_client_id4
structure. This shorthand client identifier (a client ID) is structure. This shorthand client identifier (a client ID) is
assigned by the server and should be chosen so that it will not assigned by the server and should be chosen so that it will not
conflict with a client ID previously assigned by same server, and, to conflict with a client ID previously assigned by the same server and,
the degree practicable, by other servers as well. This applies to the degree practicable, by other servers as well. This applies
across server restarts or reboots. across server restarts or reboots.
Establishment of Client ID by a new incarnation of the client also Establishment of the client ID by a new incarnation of the client
has the effect of immediately breaking any leased state that a also has the effect of immediately breaking any leased state that a
previous incarnation of the client might have had on the server, as previous incarnation of the client might have had on the server, as
opposed to forcing the new client incarnation to wait for the leases opposed to forcing the new client incarnation to wait for the leases
to expire. Breaking the lease state amounts to the server removing to expire. Breaking the lease state amounts to the server removing
all lock, share reservation, and, all delegation state not requested all locks, share reservations, and delegation states not requested
using the CLAIM_DELEGATE_PREV claim type, associated with a client using the CLAIM_DELEGATE_PREV claim type associated with a client
having the same identity. For a discussion of delegation state having the same identity. For a discussion of delegation state
recovery, see Section 10.2.1 of [RFC7530]. recovery, see Section 10.2.1 of [RFC7530].
Note that the SETCLIENTID and SETCLIENTID_CONFIRM operations have a Note that the SETCLIENTID and SETCLIENTID_CONFIRM operations have a
secondary purpose of establishing the information the server needs to secondary purpose of establishing the information the server needs to
make callbacks to the client for the purpose of supporting make callbacks to the client for the purpose of supporting
delegations. The client is able to change this information via delegations. The client is able to change this information via
SETCLIENTID and SETCLIENTID_CONFIRM within the same incarnation of SETCLIENTID and SETCLIENTID_CONFIRM within the same incarnation of
the client without causing removal of the client's leased state. the client without causing removal of the client's leased state.
Distinct servers MAY assign clientid4's independently, and will Distinct servers MAY assign clientid4s independently, and they will
generally do so. Therefore, a client has to be prepared to deal with generally do so. Therefore, a client has to be prepared to deal with
multiple instances of the same clientid4 value received on distinct multiple instances of the same clientid4 value received on distinct
IP addresses, denoting separate entities. When trunking of server IP IP addresses, denoting separate entities. When trunking of server IP
addresses is not a consideration, a client should keep track of (IP- addresses is not a consideration, a client should keep track of
address, clientid4) pairs, so that each pair is distinct. For a <IP-address, clientid4> pairs, so that each pair is distinct. For a
discussion of how to address the issue in the face of possible discussion of how to address the issue in the face of possible
trunking of server IP addresses, see Section 5.4. trunking of server IP addresses, see Section 5.4.
Owners of opens and owners of byte-range locks are separate entities Owners of opens and owners of byte-range locks are separate entities
and remain separate even if the same opaque arrays are used to and remain separate even if the same opaque arrays are used to
designate owners of each. The protocol distinguishes between open- designate owners of each. The protocol distinguishes between open-
owners (represented by open_owner4 structures) and lock-owners owners (represented by open_owner4 structures) and lock-owners
(represented by lock_owner4 structures). (represented by lock_owner4 structures).
Both sorts of owners consist of a clientid4 and an opaque owner Both sorts of owners consist of a clientid4 and an opaque owner
string. For each client, the set of distinct owner values used with string. For each client, there is a set of distinct owner values
that client constitutes the set of owners of that type, for the given used with that client which constitutes the set of known owners of
client. that type, for the given client.
Each open is associated with a specific open-owner while each byte- Each open is associated with a specific open-owner while each byte-
range lock is associated with a lock-owner and an open-owner, the range lock is associated with a lock-owner and an open-owner, the
latter being the open-owner associated with the open file under which latter being the open-owner associated with the open file under which
the LOCK operation was done. the LOCK operation was done.
When a clientid4 is presented to a server and that clientid4 is not When a clientid4 is presented to a server and that clientid4 is not
valid, the server will reject the request with the an error that valid, the server will reject the request with an error that depends
depends on the reason for clientid4 invalidity. The error on the reason for clientid4 invalidity. The error
NFS4ERR_ADMIN_REVOKED is returned when the invalidation is the result NFS4ERR_ADMIN_REVOKED is returned when the invalidation is the result
of administrative action. When the clientid4 is unrecognizable, the of administrative action. When the clientid4 is unrecognizable, the
error NFS4ERR_STALE_CLIENTID or NFS4ERR_EXPIRED may be returned. An error NFS4ERR_STALE_CLIENTID or NFS4ERR_EXPIRED may be returned. An
unrecognizable clientid4 can occur for a number of reasons: unrecognizable clientid4 can occur for a number of reasons:
o A server reboot causing loss of the server's knowledge of the o A server reboot causing loss of the server's knowledge of the
client. (Always returns NFS4ERR_STALE_CLIENTID) client. (Always returns NFS4ERR_STALE_CLIENTID.)
o Client error sending an incorrect clientid4 or a valid clientid4 o Client error sending an incorrect clientid4 or a valid clientid4
to the wrong server. (May return either error). to the wrong server. (May return either error.)
o Loss of lease state due to lease expiration. (Always returns o Loss of lease state due to lease expiration. (Always returns
NFS4ERR_EXPIRED) NFS4ERR_EXPIRED.)
o Client or server error causing the server to believe that the o Client or server error causing the server to believe that the
client has rebooted (i.e., receiving a SETCLIENTID with an client has rebooted (i.e., receiving a SETCLIENTID with an
nfs_client_id4 which has a matching id string and a non-matching nfs_client_id4 that has a matching id string and a non-matching
boot instance id as the verifier). (May return either error). boot instance id as the verifier). (May return either error.)
o Migration of all state under the associated lease causes its non- o Migration of all state under the associated lease causes its non-
existence to be recognized on the source server. (Always returns existence to be recognized on the source server. (Always returns
NFS4ERR_STALE_CLIENTID) NFS4ERR_STALE_CLIENTID.)
o Merger of state under the associated lease with another lease o Merger of state under the associated lease with another lease
under a different client ID causes the clientid4 serving as the under a different client ID causes the clientid4 serving as the
source of the merge to cease being recognized on its server. source of the merge to cease being recognized on its server.
(Always returns NFS4ERR_STALE_CLIENTID) (Always returns NFS4ERR_STALE_CLIENTID.)
In the event of a server reboot, loss of lease state due to lease In the event of a server reboot, loss of lease state due to lease
expiration, or administrative revocation of a clientid4, the client expiration, or administrative revocation of a clientid4, the client
must obtain a new clientid4 by use of the SETCLIENTID operation and must obtain a new clientid4 by use of the SETCLIENTID operation and
then proceed to any other necessary recovery for the server reboot then proceed to any other necessary recovery for the server reboot
case (See Section 9.6.2 in [RFC7530]). In cases of server or client case (see Section 9.6.2 in [RFC7530]). In cases of server or client
error resulting in a clientid4 becoming unusable, use of SETCLIENTID error resulting in a clientid4 becoming unusable, use of SETCLIENTID
to establish a new lease is desirable as well. to establish a new lease is desirable as well.
In cases in which loss of server knowledge of a clientid4 is the In cases in which loss of server knowledge of a clientid4 is the
result of migration, different recovery procedures are required. See result of migration, different recovery procedures are required. See
Section 6.1.1 for details. Note that in cases in which there is any Section 6.1.1 for details. Note that in cases in which there is any
uncertainty about which sort of handling is applicable, the uncertainty about which sort of handling is applicable, the
distinguishing characteristic is that in reboot-like cases, the distinguishing characteristic is that in reboot-like cases, the
clientid4 and all associated stateids cease to exist while in clientid4 and all associated stateids cease to exist while in
migration-related cases, the clientid4 ceases to exist while the migration-related cases, the clientid4 ceases to exist while the
stateids are still valid. stateids are still valid.
The client must also employ the SETCLIENTID operation when it The client must also employ the SETCLIENTID operation when it
receives a NFS4ERR_STALE_STATEID error using a stateid derived from receives an NFS4ERR_STALE_STATEID error using a stateid derived from
its current clientid4, since this indicates a situation, such as its current clientid4, since this indicates a situation, such as a
server reboot which has invalidated the existing clientid4 and server reboot that has invalidated the existing clientid4 and
associated stateids (see Section 9.1.5 in [RFC7530] for details). associated stateids (see Section 9.1.5 in [RFC7530] for details).
See the detailed descriptions of SETCLIENTID (in Section 8.4) and See the detailed descriptions of SETCLIENTID (in Section 8.4) and
SETCLIENTID_CONFIRM (in Section 16.34 of [RFC7530]) for a complete SETCLIENTID_CONFIRM (in Section 16.34 of [RFC7530]) for a complete
specification of these operations. specification of these operations.
5.3. Server Release of Client ID 5.3. Server Release of Client ID
If the server determines that the client holds no associated state If the server determines that the client holds no associated state
for its clientid4, the server may choose to release that clientid4. for its clientid4, the server may choose to release that clientid4.
The server may make this choice for an inactive client so that The server may make this choice for an inactive client so that
resources are not consumed by those intermittently active clients. resources are not consumed by those intermittently active clients.
If the client contacts the server after this release, the server must If the client contacts the server after this release, the server must
ensure the client receives the appropriate error so that it will use ensure the client receives the appropriate error so that it will use
the SETCLIENTID/SETCLIENTID_CONFIRM sequence to establish a new the SETCLIENTID/SETCLIENTID_CONFIRM sequence to establish a new
identity. It should be clear that the server must be very hesitant identity. It should be clear that the server must be very hesitant
to release a client ID since the resulting work on the client to to release a client ID since the resulting work on the client to
recover from such an event will be the same burden as if the server recover from such an event will be the same burden as if the server
had failed and restarted. Typically a server would not release a had failed and restarted. Typically, a server would not release a
client ID unless there had been no activity from that client for many client ID unless there had been no activity from that client for many
minutes. minutes.
Note that if the id string in a SETCLIENTID request is properly Note that if the id string in a SETCLIENTID request is properly
constructed, and if the client takes care to use the same principal constructed, and if the client takes care to use the same principal
for each successive use of SETCLIENTID, then, barring an active for each successive use of SETCLIENTID, then, barring an active
denial of service attack, NFS4ERR_CLID_INUSE should never be denial-of-service attack, NFS4ERR_CLID_INUSE should never be
returned. returned.
However, client bugs, server bugs, or perhaps a deliberate change of However, client bugs, server bugs, or perhaps a deliberate change of
the principal owner of the id string (such as may occur in the case the principal owner of the id string (such as may occur in the case
in which a client changes security flavors, and under the new flavor, in which a client changes security flavors, and under the new flavor,
there is no mapping to the previous owner) will in rare cases result there is no mapping to the previous owner) will in rare cases result
in NFS4ERR_CLID_INUSE. in NFS4ERR_CLID_INUSE.
In situations in which there is an apparent change of principal, when In situations in which there is an apparent change of principal, when
the server gets a SETCLIENTID specifying a client id string for which the server gets a SETCLIENTID specifying a client ID string for which
the server has a clientid4 that currently has no state, or for which the server has a clientid4 that currently has no state, or for which
it has state, but where the lease has expired, the server MUST allow it has state, but where the lease has expired, the server MUST allow
the SETCLIENTID, rather than returning NFS4ERR_CLID_INUSE. The the SETCLIENTID rather than returning NFS4ERR_CLID_INUSE. The server
server MUST then confirm the new client ID if followed by the MUST then confirm the new client ID if followed by the appropriate
appropriate SETCLIENTID_CONFIRM. SETCLIENTID_CONFIRM.
5.4. Client Id String Approaches 5.4. Client ID String Approaches
One particular aspect of the construction of the nfs_client_id4 One particular aspect of the construction of the nfs_client_id4
string has proved recurrently troublesome. The client has a choice string has proved recurrently troublesome. The client has a choice
of: of:
o Presenting the same id string to multiple server addresses. This o Presenting the same id string to multiple server addresses. This
is referred to as the "uniform client id string approach" and is is referred to as the "uniform client ID string approach" and is
discussed in Section 5.6. discussed in Section 5.6.
o Presenting different id strings to multiple server addresses. o Presenting different id strings to multiple server addresses.
This is referred to as the "non-uniform client id string approach" This is referred to as the "non-uniform client ID string approach"
and is discussed in Section 5.5. and is discussed in Section 5.5.
Note that implementation considerations, including compatibility with Note that implementation considerations, including compatibility with
existing servers, may make it desirable for a client to use both existing servers, may make it desirable for a client to use both
approaches, based on configuration information, such as mount approaches, based on configuration information, such as mount
options. This issue will be discussed in Section 5.7. options. This issue will be discussed in Section 5.7.
Construction of the client id string has arisen as a difficult issue Construction of the client ID string has arisen as a difficult issue
because of the way in which the NFS protocols have evolved. It is because of the way in which the NFS protocols have evolved. It is
useful to consider two points in that evolution. useful to consider two points in that evolution.
o NFSv3 as a stateless protocol had no need to identify the state o NFSv3 as a stateless protocol had no need to identify the state
shared by a particular client-server pair. (See [RFC1813]). Thus shared by a particular client-server pair (see [RFC1813]). Thus,
there was no need to consider the question of whether a set of there was no need to consider the question of whether a set of
requests come from the same client, or whether two server IP requests come from the same client or whether two server IP
addresses are connected to the same server. As the environment addresses are connected to the same server. As the environment
was one in which the user supplied the target server IP address as was one in which the user supplied the target server IP address as
part of incorporating the remote file system in the client's file part of incorporating the remote file system in the client's file
name space, there was no occasion to take note of server trunking. namespace, there was no occasion to take note of server trunking.
Within a stateless protocol, the situation was symmetrical. The Within a stateless protocol, the situation was symmetrical. The
client has no server identity information and the server has no client has no server identity information, and the server has no
client identity information. client identity information.
o NFSv4.1 is a stateful protocol with full support for client and o NFSv4.1 is a stateful protocol with full support for client and
server identity determination (See [RFC5661]). This enables the server identity determination (see [RFC5661]). This enables the
server to be aware when two requests come from the same client server to be aware when two requests come from the same client
(they are on sessions sharing a clientid4) and the client to be (they are on sessions sharing a clientid4) and the client to be
aware when two server IP addresses are connected to the same aware when two server IP addresses are connected to the same
server. Section 2.10.5.1 of [RFC5661] explains how the client is server. Section 2.10.5.1 of [RFC5661] explains how the client is
able to assure itself that the connections are to the same logical able to assure itself that the connections are to the same logical
server. server.
NFSv4.0 is unfortunately halfway between these two. It introduced NFSv4.0 is unfortunately halfway between these two. It introduced
new requirements such as the need to identify specific clients and new requirements such as the need to identify specific clients and
client instances without addressing server identity issues. The two client instances without addressing server identity issues. The two
client id string approaches have arisen in attempts to deal with the client ID string approaches have arisen in attempts to deal with the
changing requirements of the protocol as implementation has proceeded changing requirements of the protocol as implementation has
and features that were not very substantial in early implementations proceeded, and features that were not very substantial in early
of NFSv4.0, became more substantial as implementation proceeded. implementations of NFSv4.0 became more substantial as implementation
proceeded.
o In the absence of any implementation of the fs_locations-related o In the absence of any implementation of features related to
features (replication, referral, and migration), the situation is fs_locations (replication, referral, and migration), the situation
very similar to that of NFSv3 (see Section 8.1 and the subsections is very similar to that of NFSv3 (see Section 8.1 and the
within section 8.4 of [RFC7530] for discussion of these features. subsections within Section 8.4 of [RFC7530] for discussion of
In this case, locking state has been added but there is no need these features). In this case, locking state has been added, but
for concern about provision of accurate client and server identity there is no need for concern about the provision of accurate
determination. This is the situation that gave rise to the non- client and server identity determination. This is the situation
uniform client id string approach. that gave rise to the non-uniform client ID string approach.
o In the presence of replication and referrals, the client may have o In the presence of replication and referrals, the client may have
occasion to take advantage of knowledge of server trunking occasion to take advantage of knowledge of server trunking
information. Even more important, transparent state migration, by information. Even more important, transparent state migration, by
transferring state among servers, causes difficulties for the non- transferring state among servers, causes difficulties for the non-
uniform client id string approach, in that the two different uniform client ID string approach, in that the two different
client id strings sent to different IP addresses may wind up being client ID strings sent to different IP addresses may wind up being
processed by the same logical server, adding confusion. processed by the same logical server, adding confusion.
o A further consideration is that client implementations typically o A further consideration is that client implementations typically
provide NFSv4.1 by augmenting their existing NFSv4.0 provide NFSv4.1 by augmenting their existing NFSv4.0
implementation, not by providing two separate implementations. implementation, not by providing two separate implementations.
Thus the more NFSv4.0 and NFSv4.1 can work alike, the less complex Thus, the more NFSv4.0 and NFSv4.1 can work alike, the less
are clients. This is a key reason why those implementing NFSv4.0 complex the clients are. This is a key reason why those
clients might prefer using the uniform client string model, even implementing NFSv4.0 clients might prefer using the uniform client
if they have chosen not to provide fs_locations-related features string model, even if they have chosen not to provide
in their NFSv4.0 client. fs_locations-related features in their NFSv4.0 client.
Both approaches have to deal with the asymmetry in client and server Both approaches have to deal with the asymmetry in client and server
identity information between client and server. Each seeks to make identity information between client and server. Each seeks to make
the client's and the server's views match. In the process, each the client's and the server's views match. In the process, each
encounters some combination of inelegant protocol features and/or encounters some combination of inelegant protocol features and/or
implementation difficulties. The choice of which to use is up to the implementation difficulties. The choice of which to use is up to the
client implementer and the sections below try to give some useful client implementer, and the sections below try to give some useful
guidance. guidance.
5.5. Non-Uniform Client Id String Approach 5.5. Non-uniform Client ID String Approach
The non-uniform client id string approach is an attempt to handle The non-uniform client ID string approach is an attempt to handle
these matters in NFSv4.0 client implementations in as NFSv3-like a these matters in NFSv4.0 client implementations in as NFSv3-like a
way as possible. way as possible.
For a client using the non-uniform approach, all internal recording For a client using the non-uniform approach, all internal recording
of clientid4 values is to include, whether explicitly or implicitly, of clientid4 values is to include, whether explicitly or implicitly,
the server IP address so that one always has an (IP-address, the server IP address so that one always has an <IP-address,
clientid4) pair. Two such pairs from different servers are always clientid4> pair. Two such pairs from different servers are always
distinct even when the clientid4 values are the same, as they may distinct even when the clientid4 values are the same, as they may
occasionally be. In this approach, such equality is always treated occasionally be. In this approach, such equality is always treated
as simple happenstance. as simple happenstance.
Making the client id string different on different server IP Making the client ID string different on different server IP
addresses results in a situation in which a server has no way of addresses results in a situation in which a server has no way of
tying together information from the same client, when the client tying together information from the same client, when the client
accesses multiple server IP addresses. As a result, it will treat a accesses multiple server IP addresses. As a result, it will treat a
single client as multiple clients with separate leases for each single client as multiple clients with separate leases for each
server network address. Since there is no way in the protocol for server network address. Since there is no way in the protocol for
the client to determine if two network addresses are connected to the the client to determine if two network addresses are connected to the
same server, the resulting lack of knowledge is symmetrical and can same server, the resulting lack of knowledge is symmetrical and can
result in simpler client implementations in which there is a single result in simpler client implementations in which there is a single
clientid4/lease per server network address. clientid4/lease per server network address.
Support for migration, particularly with transparent state migration, Support for migration, particularly with transparent state migration,
is more complex in the case of non-uniform client id strings. For is more complex in the case of non-uniform client ID strings. For
example, migration of a lease can result in multiple leases for the example, migration of a lease can result in multiple leases for the
same client accessing the same server addresses, vitiating many of same client accessing the same server addresses, vitiating many of
the advantages of this approach. Therefore, client implementations the advantages of this approach. Therefore, client implementations
that support migration with transparent state migration are likely to that support migration with transparent state migration are likely to
experience difficulties using the non-uniform client id string experience difficulties using the non-uniform client ID string
approach, and should not do so, except where it is necessary for approach and should not do so, except where it is necessary for
compatibility with existing server implementations (For details of compatibility with existing server implementations (for details of
arranging use of multiple client id string approaches, see arranging use of multiple client ID string approaches, see
Section 5.7). Section 5.7).
5.6. Uniform Client Id String Approach 5.6. Uniform Client ID String Approach
When the client id string is kept uniform, the server has the basis When the client ID string is kept uniform, the server has the basis
to have a single clientid4/lease for each distinct client. The to have a single clientid4/lease for each distinct client. The
problem that has to be addressed is the lack of explicit server problem that has to be addressed is the lack of explicit server
identity information, which was made available in NFSv4.1. identity information, which was made available in NFSv4.1.
When the same client id string is given to multiple IP addresses, the When the same client ID string is given to multiple IP addresses, the
client can determine whether two IP addresses correspond to a single client can determine whether two IP addresses correspond to a single
server, based on the server's behavior. This is the inverse of the server, based on the server's behavior. This is the inverse of the
strategy adopted for the non-uniform approach in which different strategy adopted for the non-uniform approach in which different
server IP addresses are told about different clients, simply to server IP addresses are told about different clients, simply to
prevent a server from manifesting behavior that is inconsistent with prevent a server from manifesting behavior that is inconsistent with
there being a single server for each IP address, in line with the there being a single server for each IP address, in line with the
traditions of NFS. So, to compare: traditions of NFS. So, to compare:
o In the non-uniform approach, servers are told about different o In the non-uniform approach, servers are told about different
clients because, if the server were to use accurate information as clients because, if the server were to use accurate client
to client identity, two IP addresses on the same server would identity information, two IP addresses on the same server would
behave as if they were talking to the same client, which might behave as if they were talking to the same client, which might
prove disconcerting to a client not expecting such behavior. prove disconcerting to a client not expecting such behavior.
o In the uniform approach, the servers are told about there being a o In the uniform approach, the servers are told about there being a
single client, which is, after all, the truth. Then, when the single client, which is, after all, the truth. Then, when the
server uses this information, two IP addresses on the same server server uses this information, two IP addresses on the same server
will behave as if they are talking to the same client, and this will behave as if they are talking to the same client, and this
difference in behavior allows the client to infer the server IP difference in behavior allows the client to infer the server IP
address trunking configuration, even though NFSv4.0 does not address trunking configuration, even though NFSv4.0 does not
explicitly provide this information. explicitly provide this information.
The approach given in the section below shows one example of how The approach given in the section below shows one example of how
this might be done. this might be done.
The uniform client id string approach makes it necessary to exercise The uniform client ID string approach makes it necessary to exercise
more care in the definition of the boot instance id sent as the more care in the definition of the boot instance id sent as the
verifier field in an nfs_client_id4: verifier field in an nfs_client_id4:
o In [RFC7530], the client is told to change the verifier field o In [RFC7530], the client is told to change the verifier field
value when reboot occurs, but there is no explicit statement as to value when reboot occurs, but there is no explicit statement as to
the converse, so that any requirement to keep the verifier field the converse, so that any requirement to keep the verifier field
constant unless rebooting is only present by implication. constant unless rebooting is only present by implication.
o Many existing clients change the boot instance id every time they o Many existing clients change the boot instance id every time they
destroy and recreate the data structure that tracks an <IP- destroy and recreate the data structure that tracks an
address, clientid4> pair. This might happen if the last mount of <IP-address, clientid4> pair. This might happen if the last mount
a particular server is removed, and then a fresh mount is created. of a particular server is removed, and then a fresh mount is
Also, note that this might result in each <IP-address, clientid4> created. Also, note that this might result in each <IP-address,
pair having its own boot instance id that is independent of the clientid4> pair having its own boot instance id that is
others. independent of the others.
o Within the uniform client id string approach, an nfs_client_id4 o Within the uniform client ID string approach, an nfs_client_id4
designates a globally known client instance, so that the verifier designates a globally known client instance, so that the verifier
field should change if and only if a new client instance is field should change if and only if a new client instance is
created, typically as a result of a reboot. created, typically as a result of a reboot.
Clients using the uniform client id string approach are therefore Clients using the uniform client ID string approach are therefore
well advised to use a verifier established only once for each well advised to use a verifier established only once for each
reboot, typically at reboot time. reboot, typically at reboot time.
The following are advantages for the implementation of using the The following are advantages for the implementation of using the
uniform client id string approach: uniform client ID string approach:
o Clients can take advantage of server trunking (and clustering with o Clients can take advantage of server trunking (and clustering with
single-server-equivalent semantics) to increase bandwidth or single-server-equivalent semantics) to increase bandwidth or
reliability. reliability.
o There are advantages in state management so that, for example, one o There are advantages in state management so that, for example, one
never has a delegation under one clientid4 revoked because of a never has a delegation under one clientid4 revoked because of a
reference to the same file from the same client under a different reference to the same file from the same client under a different
clientid4. clientid4.
o The uniform client id string approach allows the server to do any o The uniform client ID string approach allows the server to do any
necessary automatic lease merger in connection with transparent necessary automatic lease merger in connection with transparent
state migration, without requiring any client involvement. This state migration, without requiring any client involvement. This
consideration is of sufficient weight to cause us to recommend use consideration is of sufficient weight to cause us to recommend use
of the uniform client id string approach for clients supporting of the uniform client ID string approach for clients supporting
transparent state migration. transparent state migration.
The following implementation considerations might cause issues for The following implementation considerations might cause issues for
client implementations. client implementations.
o This approach is considerably different from the non-uniform o This approach is considerably different from the non-uniform
approach, which most client implementations have been following. approach, which most client implementations have been following.
Until substantial implementation experience is obtained with this Until substantial implementation experience is obtained with this
approach, reluctance to embrace something so new is to be approach, reluctance to embrace something so new is to be
expected. expected.
o Mapping between server network addresses and leases is more o Mapping between server network addresses and leases is more
complicated in that it is no longer a one-to-one mapping. complicated in that it is no longer a one-to-one mapping.
Another set of relevant considerations relate to privacy concerns, Another set of relevant considerations relate to privacy concerns,
which users of the client might have in that use of the uniform which users of the client might have in that use of the uniform
client id string approach would enable multiple servers acting in client ID string approach would enable multiple servers acting in
concert to determine when multiple requests received at different concert to determine when multiple requests received at different
times, derive from the same NFSv4 client. For example, this might times derive from the same NFSv4.0 client. For example, this might
enable determination that multiple distinct user identities, in fact enable determination that multiple distinct user identities in fact
are likely to correspond to requests made by the same person, even are likely to correspond to requests made by the same person, even
when those request are directed to different servers. when those requests are directed to different servers.
How to balance these considerations depends on implementation goals. How to balance these considerations depends on implementation goals.
5.7. Mixing Client Id String Approaches 5.7. Mixing Client ID String Approaches
As noted above, a client which needs to use the uniform client id As noted above, a client that needs to use the uniform client ID
string approach (e.g., to support migration), may also need to string approach (e.g., to support migration) may also need to support
support existing servers with implementations that do not work existing servers with implementations that do not work properly in
properly in this case. this case.
Some examples of such server issues include: Some examples of such server issues include:
o Some existing NFSv4.0 server implementations of IP address o Some existing NFSv4.0 server implementations of IP address
failover depend on clients' use of a non-uniform client id string failover depend on clients' use of a non-uniform client ID string
approach. In particular, when a server supports both its own IP approach. In particular, when a server supports both its own IP
address and one failed over from a partner server, it may have address and one failed over from a partner server, it may have
separate sets of state applicable to the two IP addresses, owned separate sets of state applicable to the two IP addresses, owned
by different servers but residing on a single one. by different servers but residing on a single one.
In this situation, some servers have relied on clients' use of the In this situation, some servers have relied on clients' use of the
non-uniform client id string approach, as suggested but not non-uniform client ID string approach, as suggested but not
mandated by [RFC7530], to keep these sets of state separate, and mandated by [RFC7530], to keep these sets of state separate, and
will have problems in handling clients using the uniform client id they will have problems handling clients using the uniform client
string approach, in that such clients will see changes in trunking ID string approach, in that such clients will see changes in
relationships whenever server failover and giveback occur. trunking relationships whenever server failover and giveback
occur.
o Some existing servers incorrectly return NFS4ERR_CLID_INUSE simply o Some existing servers incorrectly return NFS4ERR_CLID_INUSE simply
because there already exists a clientid4 for the same client, because there already exists a clientid4 for the same client,
established using a different IP address. This causes difficulty established using a different IP address. This causes difficulty
for a multi-homed client using the uniform client id string for a multihomed client using the uniform client ID string
approach. approach.
Although this behavior is not correct, such servers still exist Although this behavior is not correct, such servers still exist,
and this specification should give clients guidance about dealing and this specification should give clients guidance about dealing
with the situation, as well as making the correct behavior clear. with the situation, as well as making the correct behavior clear.
In order to support use of these sorts of servers, the client can use In order to support use of these sorts of servers, the client can use
different client id string approaches for different mounts, in order different client ID string approaches for different mounts, in order
to assure that, to assure that:
o The uniform client id string approach is used when accessing o The uniform client ID string approach is used when accessing
servers that may return NFS4ERR_MOVED and the client wishes to servers that may return NFS4ERR_MOVED and when the client wishes
enable transparent state migration. to enable transparent state migration.
o The non-uniform client id string approach is used when accessing o The non-uniform client ID string approach is used when accessing
servers whose implementations make them incompatible with the servers whose implementations make them incompatible with the
uniform client id string approach. uniform client ID string approach.
Since the client cannot easily determine which of the above are true, Since the client cannot easily determine which of the above are true,
implementations are likely to rely on user-specified mount options to implementations are likely to rely on user-specified mount options to
select the appropriate approach to use, in cases in which a client select the appropriate approach to use, in cases in which a client
supports simultaneous use of multiple approaches. Choice of a supports simultaneous use of multiple approaches. Choice of a
default to use in such cases is up to the client implementation. default to use in such cases is up to the client implementation.
In the case in which the same server has multiple mounts, and both In the case in which the same server has multiple mounts, and both
approaches are specified for the same server, the client could have approaches are specified for the same server, the client could have
multiple clientid4's corresponding to the same server, one for each multiple clientid4s corresponding to the same server, one for each
approach and would then have to keep these separate. approach, and would then have to keep these separate.
5.8. Trunking Determination when Using Uniform Client Id Strings 5.8. Trunking Determination when Using Uniform Client ID Strings
This section provides an example of how trunking determination could This section provides an example of how trunking determination could
be done by a client following the uniform client id string approach be done by a client following the uniform client ID string approach
(whether this is used for all mounts or not). Clients need not (whether this is used for all mounts or not). Clients need not
follow this procedure but implementers should make sure that the follow this procedure, but implementers should make sure that the
issues dealt with by this procedure are all properly addressed. issues dealt with by this procedure are all properly addressed.
It is best to clarify here the various possible purposes of trunking It is best to clarify here the various possible purposes of trunking
determination and the corresponding requirements as to server determination and the corresponding requirements as to server
behavior. The following points should be noted: behavior. The following points should be noted:
o The primary purpose of the trunking determination algorithm is to o The primary purpose of the trunking determination algorithm is to
make sure that, if the server treats client requests on two IP make sure that, if the server treats client requests on two IP
addresses as part of the same client, the client will not be addresses as part of the same client, the client will not be
surprised and encounter disconcerting server behavior, as surprised and encounter disconcerting server behavior, as
mentioned in Section 5.6. Such behavior could occur if the client mentioned in Section 5.6. Such behavior could occur if the client
were unaware that all of its client requests for the two IP were unaware that all of its client requests for the two IP
addresses were being handled as part of a single client talking to addresses were being handled as part of a single client talking to
a single server. a single server.
o A second purpose is to be able to use knowledge of trunking o A second purpose is to be able to use knowledge of trunking
relationships for better performance, etc. relationships for better performance, etc.
o If a server were to give out distinct clientid4's in response to o If a server were to give out distinct clientid4s in response to
receiving the same nfs_client_id4 on different network addresses, receiving the same nfs_client_id4 on different network addresses,
and acted as if these were separate clients, the primary purpose and acted as if these were separate clients, the primary purpose
of trunking determination would be met, as long as the server did of trunking determination would be met, as long as the server did
not treat them as part of the same client. In this case, the not treat them as part of the same client. In this case, the
server would be acting, with regard to that client, as if it were server would be acting, with regard to that client, as if it were
two distinct servers. This would interfere with the secondary two distinct servers. This would interfere with the secondary
purpose of trunking determination but there is nothing the client purpose of trunking determination, but there is nothing the client
can do about that. can do about that.
o Suppose a server were to give such a client two different o Suppose a server were to give such a client two different
clientid4's but act as if they were one. That is the only way clientid4s but act as if they were one. That is the only way that
that the server could behave in a way that would defeat the the server could behave in a way that would defeat the primary
primary purpose of the trunking determination algorithm. purpose of the trunking determination algorithm.
Servers MUST NOT behave that way. Servers MUST NOT behave that way.
For a client using the uniform approach, clientid4 values are treated For a client using the uniform approach, clientid4 values are treated
as important information in determining server trunking patterns. as important information in determining server trunking patterns.
For two different IP addresses to return the same clientid4 value is For two different IP addresses to return the same clientid4 value is
a necessary, though not a sufficient condition for them to be a necessary, though not a sufficient condition for them to be
considered as connected to the same server. As a result, when two considered as connected to the same server. As a result, when two
different IP addresses return the same clientid4, the client needs to different IP addresses return the same clientid4, the client needs to
determine, using the procedure given below or otherwise, whether the determine, using the procedure given below or otherwise, whether the
IP addresses are connected to the same server. For such clients, all IP addresses are connected to the same server. For such clients, all
internal recording of clientid4 values needs to include, whether internal recording of clientid4 values needs to include, whether
explicitly or implicitly, identification of the server from which the explicitly or implicitly, identification of the server from which the
clientid4 was received so that one always has a (server, clientid4) clientid4 was received so that one always has a (server, clientid4)
pair. Two such pairs from different servers are always considered pair. Two such pairs from different servers are always considered
skipping to change at page 20, line 47 skipping to change at page 22, line 22
explicitly or implicitly, identification of the server from which the explicitly or implicitly, identification of the server from which the
clientid4 was received so that one always has a (server, clientid4) clientid4 was received so that one always has a (server, clientid4)
pair. Two such pairs from different servers are always considered pair. Two such pairs from different servers are always considered
distinct even when the clientid4 values are the same, as they may distinct even when the clientid4 values are the same, as they may
occasionally be. occasionally be.
In order to make this approach work, the client must have certain In order to make this approach work, the client must have certain
information accessible for each nfs_client_id4 used by the uniform information accessible for each nfs_client_id4 used by the uniform
approach (only one in general). The client needs to maintain a list approach (only one in general). The client needs to maintain a list
of all server IP addresses, together with the associated clientid4 of all server IP addresses, together with the associated clientid4
values, SETCLIENTID principals and authentication flavors. As a part values, SETCLIENTID principals, and authentication flavors. As a
of the associated data structures, there should be the ability to part of the associated data structures, there should be the ability
mark a server IP structure as having the same server as another and to mark a server IP structure as having the same server as another
to mark an IP address as currently unresolved. One way to do this is and to mark an IP address as currently unresolved. One way to do
to a allow each such entry to point to another with the pointer value this is to allow each such entry to point to another with the pointer
being one of: value being one of:
o A pointer to another entry for an IP address associated with the o A pointer to another entry for an IP address associated with the
same server, where that IP address is the first one referenced to same server, where that IP address is the first one referenced to
access that server. access that server.
o A pointer to the current entry if there is no earlier IP address o A pointer to the current entry if there is no earlier IP address
associated with the same server, i.e., where the current IP associated with the same server, i.e., where the current IP
address is the first one referenced to access that server. The address is the first one referenced to access that server. The
text below refers to such an IP address as the lead IP address for text below refers to such an IP address as the lead IP address for
a given server. a given server.
skipping to change at page 21, line 36 skipping to change at page 23, line 9
Given this apparatus, when a SETCLIENTID is done and a clientid4 Given this apparatus, when a SETCLIENTID is done and a clientid4
returned, the data structure can be searched for a matching clientid4 returned, the data structure can be searched for a matching clientid4
and if such is found, further processing can be done to determine and if such is found, further processing can be done to determine
whether the clientid4 match is accidental, or the result of trunking. whether the clientid4 match is accidental, or the result of trunking.
In this algorithm, when SETCLIENTID is done initially, it will use In this algorithm, when SETCLIENTID is done initially, it will use
the common nfs_client_id4 and specify the current target IP address the common nfs_client_id4 and specify the current target IP address
as callback.cb_location within the callback parameters. We call the as callback.cb_location within the callback parameters. We call the
clientid4 and SETCLIENTID verifier returned by this operation XC and clientid4 and SETCLIENTID verifier returned by this operation XC and
XV. XV, respectively.
This choice of callback parameters is provisional and reflects the This choice of callback parameters is provisional and reflects the
client's preferences in the event that the IP address is not trunked client's preferences in the event that the IP address is not trunked
with other IP addresses. The algorithm is constructed so that only with other IP addresses. The algorithm is constructed so that only
the appropriate callback parameters, reflecting observed trunking the appropriate callback parameters, reflecting observed trunking
patterns is actually confirmed. patterns, are actually confirmed.
Note that when the client has done previous SETCLIENTID's, to any IP Note that when the client has done previous SETCLIENTIDs to any IP
addresses, with more than one principal or authentication flavor, one addresses, with more than one principal or authentication flavor, one
has the possibility of receiving NFS4ERR_CLID_INUSE, since it is not has the possibility of receiving NFS4ERR_CLID_INUSE, since it is not
yet knows which of the connections with existing IP addresses might yet known which of the connections with existing IP addresses might
be trunked with the current one. In the event that the SETCLIENTID be trunked with the current one. In the event that the SETCLIENTID
fails with NFS4ERR_CLID_INUSE, one must try all other combinations of fails with NFS4ERR_CLID_INUSE, one must try all other combinations of
principals and authentication flavors currently in use and eventually principals and authentication flavors currently in use, and
one will be correct and not return NFS4ERR_CLID_INUSE. eventually one will be correct and not return NFS4ERR_CLID_INUSE.
Note that at this point, no SETCLIENTID_CONFIRM has yet been done. Note that at this point, no SETCLIENTID_CONFIRM has yet been done.
This is because the SETCLIENTID just done has either established a This is because the SETCLIENTID just done has either established a
new clientid4 on a previously unknown server or changed the callback new clientid4 on a previously unknown server or changed the callback
parameters on a clientid4 associated with some already known server. parameters on a clientid4 associated with some already known server.
Given it is undesirable to confirm something that should not happen, Given it is undesirable to confirm something that should not happen,
what is to be done next depends on information about existing what is to be done next depends on information about existing
clientid4's. clientid4s.
o If no matching clientid4 is found, the IP address X and clientid4 o If no matching clientid4 is found, the IP address X and clientid4
XC are added to the list and considered as having no existing XC are added to the list and considered as having no existing
known IP addresses trunked with it. The IP address is marked as a known IP addresses trunked with it. The IP address is marked as a
lead IP address for a new server. A SETCLIENTID_CONFIRM is done lead IP address for a new server. A SETCLIENTID_CONFIRM is done
using XC and XV. using XC and XV.
o If a matching clientid4 is found which is marked unresolved, o If a matching clientid4 is found that is marked unresolved,
processing on the new IP address is suspended. In order to processing on the new IP address is suspended. In order to
simplify processing, there can only be one unresolved IP address simplify processing, there can only be one unresolved IP address
for any given clientid4. for any given clientid4.
o If one or more matching clientid4's is found, none of which is o If one or more matching clientid4s are found, none of which are
marked unresolved, the new IP address X is entered and marked marked unresolved, the new IP address X is entered and marked
unresolved. A SETCLIENTID_CONFIRM is done to X using XC and XV. unresolved. A SETCLIENTID_CONFIRM is done to X using XC and XV.
After applying the steps below to each of the lead IP addresses When, as a result of encountering the last of the three cases shown
with a matching clientid4, the address will have been resolved: It above, an unresolved IP address exists, further processing is
may have been determined to be part of an already known server as required. After applying the steps below to each of the lead IP
a new IP address to be added to an existing set of IP addresses addresses with a matching clientid4, the address will have been
for that server. Otherwise, it will be recognized as a new resolved: It may have been determined to be part of an already known
server. At the point at which this determination is made, the server as a new IP address to be added to an existing set of IP
unresolved indication is cleared and any suspended SETCLIENTID addresses for that server. Otherwise, it will be recognized as a new
processing is restarted server. At the point at which this determination is made, the
unresolved indication is cleared and any suspended SETCLIENTID
processing is restarted.
For each lead IP address IPn with a clientid4 matching XC, the For each lead IP address IPn with a clientid4 matching XC, the
following steps are done. Because the RPC to do a SETCLIENTID could following steps are done. Because the Remote Procedure Call (RPC) to
take considerable time, it is desirable for the client to perform do a SETCLIENTID could take considerable time, it is desirable for
these operations in parallel. Note that because the clientid4 is a the client to perform these operations in parallel. Note that
64-bit value, the number of such IP addresses that would need to be because the clientid4 is a 64-bit value, the number of such IP
tested is expected to be quite small, even when the client is addresses that would need to be tested is expected to be quite small,
interacting with many NFSv4.0 servers. Thus, while parallel even when the client is interacting with many NFSv4.0 servers. Thus,
processing is desirable, it is not necessary. while parallel processing is desirable, it is not necessary.
o If the principal for IPn does not match that for X, the IP address o If the principal for IPn does not match that for X, the IP address
is skipped, since it is impossible for IPn and X to be trunked in is skipped, since it is impossible for IPn and X to be trunked in
these circumstances. If the principal does match but the these circumstances. If the principal does match but the
authentication flavor does not, the authentication flavor already authentication flavor does not, the authentication flavor already
used should be used for address X as well. This will avoid any used should be used for address X as well. This will avoid any
possibility that NFS4ERR_CLID_INUSE will be returned for the possibility that NFS4ERR_CLID_INUSE will be returned for the
SETCLIENTID and SETCLIENTID_CONFIRM to be done below, as long as SETCLIENTID and SETCLIENTID_CONFIRM to be done below, as long as
the server(s) at IP addresses IPn and X are correctly implemented. the server(s) at IP addresses IPn and X is correctly implemented.
o A SETCLIENTID is done to update the callback parameters to reflect o A SETCLIENTID is done to update the callback parameters to reflect
the possibility that X will be marked as associated with the the possibility that X will be marked as associated with the
server whose lead IP address is IPn. The specific callback server whose lead IP address is IPn. The specific callback
parameters chosen, in terms of cb_client4 and callback_ident, are parameters chosen, in terms of cb_client4 and callback_ident, are
up to the client and should reflect its preferences as to callback up to the client and should reflect its preferences as to callback
handling for the common clientid4, in the event that X and IPn are handling for the common clientid4, in the event that X and IPn are
trunked together. When a SETCLIENTID is done on IP address IPn, a trunked together. When a SETCLIENTID is done on IP address IPn, a
setclientid_confirm value (in the form of a verifier4) is setclientid_confirm value (in the form of a verifier4) is
returned, which will be referred to as SCn. returned, which will be referred to as SCn.
Note that the NFSv4.0 specification requires the server to make Note that the NFSv4.0 specification requires the server to make
sure that such verifiers are very unlikely to be regenerated. sure that such verifiers are very unlikely to be regenerated.
Given that it is already highly unlikely that the clientid4 XC is Given that it is already highly unlikely that the clientid4 XC is
duplicated by distinct servers, the probability that SCn is duplicated by distinct servers, the probability that SCn is
duplicated as well has to be considered vanishingly small. Note duplicated as well has to be considered vanishingly small. Note
also that the callback update procedure can be repeated multiple also that the callback update procedure can be repeated multiple
times to reduce the probability of spurious matches further. times to reduce the probability of further spurious matches.
o The setclientid_confirm value SCn is saved for later use in o The setclientid_confirm value SCn is saved for later use in
confirming the SETCLIENTID done to IPn. confirming the SETCLIENTID done to IPn.
Once the SCn values are gathered up by the procedure above, they are Once the SCn values are gathered up by the procedure above, they are
each tested by being used as the verifier for a SETCLIENTID_CONFIRM each tested by being used as the verifier for a SETCLIENTID_CONFIRM
operation directed to the original IP address X, whose trunking operation directed to the original IP address X, whose trunking
relationships are to be determined. These RPC operations may be done relationships are to be determined. These RPC operations may be done
in parallel. in parallel.
There are a number of things that should be noted at this point. There are a number of things that should be noted at this point.
o That the SETCLIENTID operations done on the various IPn addresses o The SETCLIENTID operations done on the various IPn addresses in
in the procedure above will never be confirmed by the procedure above will never be confirmed by SETCLIENTID_CONFIRM
SETCLIENTID_CONFIRM operations directed to the various IPn operations directed to the various IPn addresses. If these
addresses. If these callback updates are to be confirmed, they callback updates are to be confirmed, they will be confirmed by
will be confirmed by SETCLIENTID_CONFIRM operations directed at SETCLIENTID_CONFIRM operations directed at the original IP address
the original IP address X, which can only happen if SCn was X, which can only happen if SCn was generated by an IPn that was
generated by an IPn which was trunked with X, allowing the trunked with X, allowing the SETCLIENTID to be successfully
SETCLIENTID to be successfully confirmed, and allowing us to infer confirmed and allowing us to infer the existence of that trunking
the existence of that trunking relationship. relationship.
o That the number of successful SETCLIENTID_CONFIRM operations done o The number of successful SETCLIENTID_CONFIRM operations done
should never be more than one. If both SCn and SCm are accepted should never be more than one. If both SCn and SCm are accepted
by X, then it indicates that both IPn and IPm are trunked with X, by X, then it indicates that both IPn and IPm are trunked with X,
but that is only possible if IPn and IPm are trunked together. but that is only possible if IPn and IPm are trunked together.
Since these two addresses were earlier recognized as not trunked Since these two addresses were earlier recognized as not trunked
together, this should be impossible, if the servers in question together, this should be impossible, if the servers in question
are implemented correctly. are implemented correctly.
Further processing depends on the success or failure of the various Further processing depends on the success or failure of the various
SETCLIENTD_CONFIRM operations done in the step above. SETCLIENTD_CONFIRM operations done in the step above.
o If the setclientid_confirm value generated by a particular IPn is o If the setclientid_confirm value generated by a particular IPn is
accepted on X then X and IPn are recognized as connected to the accepted on X, then X and IPn are recognized as connected to the
same server and the entry for X is marked as associated with IPn. same server, and the entry for X is marked as associated with IPn.
o If none of the confirm operations are accepted, then X is o If none of the confirm operations are accepted, then X is
recognized as a distinct server. Its callback parameters will recognized as a distinct server. Its callback parameters will
remain as the ones established by the original SETCLIENTID. remain as the ones established by the original SETCLIENTID.
In either of the cases, the entry is considered resolved and In either of the cases, the entry is considered resolved and
processing can be restarted for IP addresses whose clientid4 matched processing can be restarted for IP addresses whose clientid4 matched
XC but whose resolution had been deferred. XC but whose resolution had been deferred.
The procedure described above must be performed so as to exclude the The procedure described above must be performed so as to exclude the
possibility that multiple SETCLIENTID's, done to different server IP possibility that multiple SETCLIENTIDs done to different server IP
addresses and returning the same clientid4 might "race" in such a addresses and returning the same clientid4 might "race" in such a
fashion that there is no explicit determination of whether they fashion that there is no explicit determination of whether they
correspond to the same server. The following possibilities for correspond to the same server. The following possibilities for
serialization are all valid and implementers may choose among them serialization are all valid, and implementers may choose among them
based on a tradeoff between performance and complexity. They are based on a tradeoff between performance and complexity. They are
listed in order of increasing parallelism: listed in order of increasing parallelism:
o An NFSv4.0 client might serialize all instances of SETCLIENTID/ o An NFSv4.0 client might serialize all instances of SETCLIENTID/
SETCLIENTID_CONFIRM processing, either directly or by serializing SETCLIENTID_CONFIRM processing, either directly or by serializing
mount operations involving use of NFSv4.0. While doing so will mount operations involving use of NFSv4.0. While doing so will
prevent the races mentioned above, this degree of serialization prevent the races mentioned above, this degree of serialization
can cause performance issues when there is a high volume of mount can cause performance issues when there is a high volume of mount
operations. operations.
skipping to change at page 25, line 4 skipping to change at page 26, line 27
This prevents the races mentioned above, without adding to delay This prevents the races mentioned above, without adding to delay
except when trunking determination is common. except when trunking determination is common.
o One might avoid much of the serialization implied above, by o One might avoid much of the serialization implied above, by
allowing trunking determination for distinct clientid4 values to allowing trunking determination for distinct clientid4 values to
happen in parallel, with serialization of trunking determination happen in parallel, with serialization of trunking determination
happening independently for each distinct clientid4 value. happening independently for each distinct clientid4 value.
The procedure above has made no explicit mention of the possibility The procedure above has made no explicit mention of the possibility
that server reboot can occur at any time. To address this that server reboot can occur at any time. To address this
possibility the client should make sure the following steps are possibility, the client should make sure the following steps are
taken: taken:
o When a SETCLIENTID_CONFIRM is rejected by a given IPn, the client o When a SETCLIENTID_CONFIRM is rejected by a given IPn, the client
should be aware of the possibility that the rejection is due to XC should be aware of the possibility that the rejection is due to XC
(rather than XV) being invalid. This situation can be addressed (rather than XV) being invalid. This situation can be addressed
by doing a RENEW specifying XC directed to the IP address X. If by doing a RENEW specifying XC directed to the IP address X. If
that operation succeeds, then the rejection is to be acted on that operation succeeds, then the rejection is to be acted on
normally since either XV is invalid on IPn or XC has become normally since either XV is invalid on IPn or XC has become
invalid on IPn while it is valid on X, showing that IPn and X are invalid on IPn while it is valid on X, showing that IPn and X are
not trunked. If, on the other hand, XC is not valid on X, then not trunked. If, on the other hand, XC is not valid on X, then
the trunking detection process should be restarted once a new the trunking detection process should be restarted once a new
client ID is established on X. client ID is established on X.
o In the event of a reboot detected on any server lead IP, the set o In the event of a reboot detected on any server-lead IP, the set
of IP addresses associated with the server should not change and of IP addresses associated with the server should not change, and
state should be re-established for the lease as a whole, using all state should be re-established for the lease as a whole, using all
available connected server IP addresses. It is prudent to verify available connected server IP addresses. It is prudent to verify
connectivity by doing a RENEW using the new clientid4 on each such connectivity by doing a RENEW using the new clientid4 on each such
server address before using it, however. server address before using it, however.
Another situation not discussed explicitly above is the possibility Another situation not discussed explicitly above is the possibility
that a SETCLIENTID done to one of the IPn addresses might take so that a SETCLIENTID done to one of the IPn addresses might take so
long that it is necessary to time out the operation, to prevent long that it is necessary to time out the operation, to prevent
unacceptably delaying the MOUNT operation. One simple possibility is unacceptably delaying the MOUNT operation. One simple possibility is
to simply fail the MOUNT at this point. Because the average number to simply fail the MOUNT at this point. Because the average number
of IP addresses that might have to be tested is quite small, this of IP addresses that might have to be tested is quite small, this
will not greatly increase the probability of MOUNT failure. Other will not greatly increase the probability of MOUNT failure. Other
possible approaches are: possible approaches are:
o If the IPn has sufficient state in existence, the existing o If the IPn has sufficient state in existence, the existing
stateids and sequence values might be validated by being used on stateids and sequence values might be validated by being used on
IP address X. In the event of success, X and IPn should be IP address X. In the event of success, X and IPn should be
considered trunked together. considered trunked together.
What constitutes "sufficient" state in this context is an What constitutes "sufficient" state in this context is an
implementation decision which is affected by the implementer's implementation decision that is affected by the implementer's
willingness to fail the MOUNT in an uncertain case, and the willingness to fail the MOUNT in an uncertain case and the
strength of the state verification procedure implemented. strength of the state verification procedure implemented.
o If IPn has no locking state in existence, X could be recorded as a o If IPn has no locking state in existence, X could be recorded as a
lead IP address on a provisional basis, subject to trunking being lead IP address on a provisional basis, subject to trunking being
tested again, once IPn starts becoming responsive. To avoid tested again, once IPn starts becoming responsive. To avoid
confusion between IPn and X, and the need to merge distinct state confusion between IPn and X, and the need to merge distinct state
corpora for X and IPn at a later point, this retest of trunking corpora for X and IPn at a later point, this retest of trunking
should occur after RENEWs on IPn are responded to and before should occur after RENEWs on IPn are responded to and before
establishing any new state for either IPn as a separate server or establishing any new state for either IPn as a separate server or
for IPn considered as a server address trunked with X. for IPn considered as a server address trunked with X.
skipping to change at page 26, line 20 skipping to change at page 27, line 43
trunked. trunked.
This choice has a lot of complexity associated with it, and it is This choice has a lot of complexity associated with it, and it is
likely that few implementations will use it. When the set of likely that few implementations will use it. When the set of
locking state on IPn is small (e.g., a single stateid) but not locking state on IPn is small (e.g., a single stateid) but not
empty, most client implementations are likely to either fail the empty, most client implementations are likely to either fail the
MOUNT or implement a more stringent verification procedure using MOUNT or implement a more stringent verification procedure using
the existing stateid on IPn as a basis to generate further state the existing stateid on IPn as a basis to generate further state
as raw material for the trunking verification process. as raw material for the trunking verification process.
5.9. Client Id String Construction Details 5.9. Client ID String Construction Details
This section gives more detailed guidance on client id string This section gives more detailed guidance on client ID string
construction. The guidance in this section will cover cases in which construction. The guidance in this section will cover cases in which
either the uniform or the non-uniform approach to the client id either the uniform or the non-uniform approach to the client ID
string is used. string is used.
Note that among the items suggested for inclusion, there are many Note that among the items suggested for inclusion, there are many
that may conceivably change. In order for the client id string to that may conceivably change. In order for the client ID string to
remain valid in such circumstances, the client SHOULD either: remain valid in such circumstances, the client SHOULD either:
o Use a saved copy of such value, rather than the changeable value o Use a saved copy of such value rather than the changeable value
itself. itself, or
o Save the constructed client id string, rather than constructing it o Save the constructed client ID string rather than constructing it
anew at SETCLIENTID time, based on unchangeable parameters and anew at SETCLIENTID time, based on unchangeable parameters and
saved copies of changeable data items. saved copies of changeable data items.
A file is not always a valid choice to store such information, given A file is not always a valid choice to store such information, given
the existence of diskless clients. In such situations, whatever the existence of diskless clients. In such situations, whatever
facilities exist for a client to store configuration information such facilities exist for a client to store configuration information such
as boot arguments should be used. as boot arguments should be used.
Given the considerations listed in Section 5.2.1, an id string would Given the considerations listed in Section 5.2.1, an id string would
be one that includes as its basis: be one that includes as its basis:
o An identifier uniquely associated with the node on which the o An identifier uniquely associated with the node on which the
client is running. client is running.
o For a user level NFSv4.0 client, it should contain additional o For a user-level NFSv4.0 client, it should contain additional
information to distinguish the client from a kernel-based client information to distinguish the client from a kernel-based client
and from other user-level clients running on the same node, such and from other user-level clients running on the same node, such
as a universally unique identifier (UUID). as a universally unique identifier (UUID).
o Where the non-uniform approach is to be used, the IP address of o Where the non-uniform approach is to be used, the IP address of
the server. the server.
o Additional information that tends to be unique, such as one or o Additional information that tends to be unique, such as one or
more of: more of:
skipping to change at page 27, line 26 skipping to change at page 29, line 5
* A true random number, generally established once and saved. * A true random number, generally established once and saved.
With regard to the identifier associated with the node on which the With regard to the identifier associated with the node on which the
client is running, the following possibilities are likely candidates. client is running, the following possibilities are likely candidates.
o The client machine's serial number. o The client machine's serial number.
o The client's IP address. Note that this SHOULD be treated as a o The client's IP address. Note that this SHOULD be treated as a
changeable value. changeable value.
o A MAC address. Note that this also should be considered a o A Media Access Control (MAC) address. Note that this also should
changeable value because of the possibility of configuration be considered a changeable value because of the possibility of
changes. configuration changes.
Privacy concerns may be an issue if some of the items above (e.g., Privacy concerns may be an issue if some of the items above (e.g.,
machine serial number, MAC address) are used. When it is necessary machine serial number and MAC address) are used. When it is
to use such items to ensure uniqueness, application of a one-way hash necessary to use such items to ensure uniqueness, application of a
function is desirable. When the non-uniform approach is used, that one-way hash function is desirable. When the non-uniform approach is
hash function should be applied to all of the components chosen as a used, that hash function should be applied to all of the components
unit rather that to particular individual elements. chosen as a unit rather than to particular individual elements.
6. Locking and Multi-Server Namespace 6. Locking and Multi-Server Namespace
This chapter contains a replacement for section 9.14, "Migration, This section contains a replacement for Section 9.14 of [RFC7530],
Replication, and State", in [RFC7530]. "Migration, Replication, and State".
The replacement is in Section 6.1 and supersedes the replaced The replacement is in Section 6.1 and supersedes the replaced
section. section.
The changes made can be briefly summarized as follows: The changes made can be briefly summarized as follows:
o Adding text to address the case of stateid conflict on migration. o Adding text to address the case of stateid conflict on migration.
o Specifying that when leases are moved, as a result of file system o Specifying that when leases are moved, as a result of file system
migration, they are to be merged with leases on the destination migration, they are to be merged with leases on the destination
skipping to change at page 28, line 38 skipping to change at page 30, line 16
client and server (i.e., locks, leases, stateids, and client IDs) is client and server (i.e., locks, leases, stateids, and client IDs) is
as described below. The handling differs between migration and as described below. The handling differs between migration and
replication. replication.
If a server replica or a server immigrating a file system agrees to, If a server replica or a server immigrating a file system agrees to,
or is expected to, accept opaque values from the client that or is expected to, accept opaque values from the client that
originated from another server, then it is a wise implementation originated from another server, then it is a wise implementation
practice for the servers to encode the "opaque" values in network practice for the servers to encode the "opaque" values in network
byte order (i.e., in a big-endian format). When doing so, servers byte order (i.e., in a big-endian format). When doing so, servers
acting as replicas or immigrating file systems will be able to parse acting as replicas or immigrating file systems will be able to parse
values like stateids, directory cookies, filehandles, etc. even if values like stateids, directory cookies, filehandles, etc., even if
their native byte order is different from that of other servers their native byte order is different from that of other servers
cooperating in the replication and migration of the file system. cooperating in the replication and migration of the file system.
6.1.1. Migration and State 6.1.1. Migration and State
In the case of migration, the servers involved in the migration of a In the case of migration, the servers involved in the migration of a
file system should transfer all server state associated with the file system should transfer all server state associated with the
migrating file system from source to the destination server. If migrating file system from source to the destination server. If
state is transferred, this MUST be done in a way that is transparent state is transferred, this MUST be done in a way that is transparent
to the client. This state transfer will ease the client's transition to the client. This state transfer will ease the client's transition
when a file system migration occurs. If the servers are successful when a file system migration occurs. If the servers are successful
in transferring all state, the client will continue to use stateids in transferring all state, the client will continue to use stateids
assigned by the original server. Therefore the new server must assigned by the original server. Therefore, the new server must
recognize these stateids as valid and treat them as representing the recognize these stateids as valid and treat them as representing the
same locks as they did on the source server. same locks as they did on the source server.
In this context, the phrase "the same locks" means: In this context, the phrase "the same locks" means that:
o That they are associated with the same file o They are associated with the same file.
o That they represent the same types of locks, whether opens, o They represent the same types of locks, whether opens,
delegations, advisory byte-range locks, or mandatory byte-range delegations, advisory byte-range locks, or mandatory byte-range
locks. locks.
o That they have the same lock particulars, including such things as o They have the same lock particulars, including such things as
access modes, deny modes, and byte ranges. access modes, deny modes, and byte ranges.
o That they are associated with the same owner string(s). o They are associated with the same owner string(s).
If transferring stateids from server to server would result in a If transferring stateids from server to server would result in a
conflict for an existing stateid for the destination server with the conflict for an existing stateid for the destination server with the
existing client, transparent state migration MUST NOT happen for that existing client, transparent state migration MUST NOT happen for that
client. Servers participating in using transparent state migration client. Servers participating in using transparent state migration
should co-ordinate their stateid assignment policies to make this should coordinate their stateid assignment policies to make this
situation unlikely or impossible. The means by which this might be situation unlikely or impossible. The means by which this might be
done, like all of the inter-server interactions for migration, are done, like all of the inter-server interactions for migration, are
not specified by the NFS version 4.0 protocol (either in [RFC7530] or not specified by the NFS version 4.0 protocol (neither in [RFC7530]
this update). nor this update).
A client may determine the disposition of migrated state by using a A client may determine the disposition of migrated state by using a
stateid associated with the migrated state on the new server. stateid associated with the migrated state on the new server.
o If the stateid is not valid and an error NFS4ERR_BAD_STATEID is o If the stateid is not valid and an error NFS4ERR_BAD_STATEID is
received, either transparent state migration has not occurred or received, either transparent state migration has not occurred or
the state was purged due to a mismatch in the verifier (i.e., the the state was purged due to a mismatch in the verifier (i.e., the
boot instance id). boot instance id).
o If the stateid is valid, transparent state migration has occurred. o If the stateid is valid, transparent state migration has occurred.
skipping to change at page 30, line 9 skipping to change at page 31, line 34
server implements migration and it does not transfer state server implements migration and it does not transfer state
information, it MUST provide a file-system-specific grace period, to information, it MUST provide a file-system-specific grace period, to
allow clients to reclaim locks associated with files in the migrated allow clients to reclaim locks associated with files in the migrated
file system. If it did not do so, clients would have to re-obtain file system. If it did not do so, clients would have to re-obtain
locks, with no assurance that a conflicting lock was not granted locks, with no assurance that a conflicting lock was not granted
after the file system was migrated and before the lock was re- after the file system was migrated and before the lock was re-
obtained. obtained.
In the case of migration without state transfer, when the client In the case of migration without state transfer, when the client
presents state information from the original server (e.g., in a RENEW presents state information from the original server (e.g., in a RENEW
op or a READ op of zero length), the client must be prepared to operation or a READ operation of zero length), the client must be
receive either NFS4ERR_STALE_CLIENTID or NFS4ERR_BAD_STATEID from the prepared to receive either NFS4ERR_STALE_CLIENTID or
new server. The client should then recover its state information as NFS4ERR_BAD_STATEID from the new server. The client should then
it normally would in response to a server failure. The new server recover its state information as it normally would in response to a
must take care to allow for the recovery of state information as it server failure. The new server must take care to allow for the
would in the event of server restart. recovery of state information as it would in the event of server
restart.
In those situations in which state has not been transferred, as shown In those situations in which state has not been transferred, as shown
by a return of NFS4ERR_BAD_STATEID, the client may attempt to reclaim by a return of NFS4ERR_BAD_STATEID, the client may attempt to reclaim
locks in order to take advantage of cases in which the destination locks in order to take advantage of cases in which the destination
server has set up a file-system-specific grace period in support of server has set up a file-system-specific grace period in support of
the migration. the migration.
6.1.1.1. Migration and Client IDs 6.1.1.1. Migration and Client IDs
Handling of clientid4 values is similar to that for stateids. The handling of clientid4 values is similar to that for stateids.
However, there are some differences that derive from the fact that a However, there are some differences that derive from the fact that a
clientid4 is an object which spans multiple file systems while a clientid4 is an object that spans multiple file systems while a
stateid is inherently limited to a single file system. stateid is inherently limited to a single file system.
The clientid4 and nfs_client_id4 information (id string and boot The clientid4 and nfs_client_id4 information (id string and boot
instance id) will be transferred with the rest of the state instance id) will be transferred with the rest of the state
information and the destination server should use that information to information, and the destination server should use that information
determine appropriate clientid4 handling. Although the destination to determine appropriate clientid4 handling. Although the
server may make state stored under an existing lease available under destination server may make state stored under an existing lease
the clientid4 used on the source server, the client should not assume available under the clientid4 used on the source server, the client
that this is always so. In particular, should not assume that this is always so. In particular,
o If there is an existing lease with an nfs_client_id4 that matches o If there is an existing lease with an nfs_client_id4 that matches
a migrated lease (same id string and verifier), the server SHOULD a migrated lease (same id string and verifier), the server SHOULD
merge the two, making the union of the sets of stateids available merge the two, making the union of the sets of stateids available
under the clientid4 for the existing lease. As part of the lease under the clientid4 for the existing lease. As part of the lease
merger, the expiration time of the lease will reflect renewal done merger, the expiration time of the lease will reflect renewal done
within either of the ancestor leases (and so will reflect the within either of the ancestor leases (and so will reflect the
latest of the renewals). latest of the renewals).
o If there is an existing lease with an nfs_client_id4 that o If there is an existing lease with an nfs_client_id4 that
partially matches a migrated lease (same id string and a different partially matches a migrated lease (same id string and a different
(boot) verifier), the server MUST eliminate one of the two, (boot) verifier), the server MUST eliminate one of the two,
possibly invalidating one of the ancestor clientid4's. Since boot possibly invalidating one of the ancestor clientid4s. Since boot
instance id's are not ordered, the later lease renewal time will instance ids are not ordered, the later lease renewal time will
prevail. prevail.
o If the destination server already has the transferred clientid4 in o If the destination server already has the transferred clientid4 in
use for another purpose, it is free to substitute a different use for another purpose, it is free to substitute a different
clientid4 and associate that with the transferred nfs_client_id4. clientid4 and associate that with the transferred nfs_client_id4.
When leases are not merged, the transfer of state should result in When leases are not merged, the transfer of state should result in
creation of a confirmed client record with empty callback information creation of a confirmed client record with empty callback information
but matching the {v, x, c} with v and x derived from the transferred but matching the {v, x, c} with v and x derived from the transferred
client information and c chosen by the destination server. For a client information and c chosen by the destination server. For a
description of this notation, see Section 8.4.5 description of this notation, see Section 8.4.5
In such cases, the client SHOULD re-establish new callback In such cases, the client SHOULD re-establish new callback
information with the new server as soon as possible, according to information with the new server as soon as possible, according to
sequences described in sections "Operation 35: SETCLIENTID - sequences described in sections "Operation 35: SETCLIENTID --
Negotiate Client ID" and "Operation 36: SETCLIENTID_CONFIRM - Confirm Negotiate Client ID" and "Operation 36: SETCLIENTID_CONFIRM --
Client ID". This ensures that server operations are not delayed due Confirm Client ID". This ensures that server operations are not
to an inability to recall delegations. The client can determine the delayed due to an inability to recall delegations and prevents the
new clientid4 (the value c) from the response to SETCLIENTID. unwanted revocation of existing delegations. The client can
determine the new clientid4 (the value c) from the response to
SETCLIENTID.
The client can use its own information about leases with the The client can use its own information about leases with the
destination server to see if lease merger should have happened. When destination server to see if lease merger should have happened. When
there is any ambiguity, the client MAY use the above procedure to set there is any ambiguity, the client MAY use the above procedure to set
the proper callback information and find out, as part of the process, the proper callback information and find out, as part of the process,
the correct value of its clientid4 with respect to the server in the correct value of its clientid4 with respect to the server in
question. question.
6.1.1.2. Migration and State Owner Information 6.1.1.2. Migration and State Owner Information
In addition to stateids, the locks they represent, and client In addition to stateids, the locks they represent, and client
identity information, servers also need to transfer information identity information, servers also need to transfer information
related to the current status of open-owners and lock-owners. related to the current status of open-owners and lock-owners.
This information includes: This information includes:
o The sequence number of the last operation associated with the o The sequence number of the last operation associated with the
particular owner. particular owner.
o Information regarding the results of the last operation, o Sufficient information regarding the results of the last operation
sufficient to allow reissued operations to be correctly responded to allow reissued operations to be correctly responded to.
to.
When individual open-owners and lock-owners have only been used in When individual open-owners and lock-owners have only been used in
connection with a particular file system, the server SHOULD transfer connection with a particular file system, the server SHOULD transfer
this information together with the lock state. The owner ceases to this information together with the lock state. The owner ceases to
exist on the source server and is reconstituted on the destination exist on the source server and is reconstituted on the destination
server. This will happen in the case of clients which have been server. This will happen in the case of clients that have been
written to isolate each owner to a specific filesystem, but may written to isolate each owner to a specific file system, but it may
happen for other clients as well. happen for other clients as well.
Note that when servers take this approach for all owners whose state Note that when servers take this approach for all owners whose state
is limited to the particular file system being migrated, doing so is limited to the particular file system being migrated, doing so
will not cause difficulties for clients not adhering to an approach will not cause difficulties for clients not adhering to an approach
in which owners are isolated to particular file systems. As long as in which owners are isolated to particular file systems. As long as
the client recognizes the loss of transferred state, the protocol the client recognizes the loss of transferred state, the protocol
allows the owner in question to disappear and the client may have to allows the owner in question to disappear, and the client may have to
deal with an owner confirmation request that would not have occurred deal with an owner confirmation request that would not have occurred
in the absence of the migration. in the absence of the migration.
When migration occurs and the source server discovers an owner whose When migration occurs and the source server discovers an owner whose
state includes the migrated file system but other file systems as state includes the migrated file system but other file systems as
well, it cannot transfer the associated owner state. Instead, the well, it cannot transfer the associated owner state. Instead, the
existing owner state stays in place but propagation of owner state is existing owner state stays in place, but propagation of owner state
done as specified below is done as specified below:
o When the current seqid for an owner represents an operation o When the current seqid for an owner represents an operation
associated with the file system being migrated, owner status associated with the file system being migrated, owner status
SHOULD be propagated to the destination file system. SHOULD be propagated to the destination file system.
o When the current seqid for an owner does not represent an o When the current seqid for an owner does not represent an
operation associated with the file system being migrated, owner operation associated with the file system being migrated, owner
status MAY be propagated to the destination file system. status MAY be propagated to the destination file system.
o When the owner in question has never been used for an operation o When the owner in question has never been used for an operation
involving the migrated file system, the owner information SHOULD involving the migrated file system, the owner information SHOULD
NOT be propagated to the destination file system. NOT be propagated to the destination file system.
Note that a server may obey all of the conditions above without the Note that a server may obey all of the conditions above without the
overhead of keeping track of set of file systems that any particular overhead of keeping track of a set of file systems that any
owner has been associated with. Consider a situation in which the particular owner has been associated with. Consider a situation in
source server has decided to keep lock-related state associated with which the source server has decided to keep lock-related state
a file system fixed, preparatory to propagating it to the destination associated with a file system fixed, preparatory to propagating it to
file system. If a client is free to create new locks associated with the destination file system. If a client is free to create new locks
existing owners on other file systems, the owner information may be associated with existing owners on other file systems, the owner
propagated to the destination file system, even though, at the time information may be propagated to the destination file system, even
the file system migration is recognized by the client to have though, at the time the file system migration is recognized by the
occurred, the last operation associated with the owner may not be client to have occurred, the last operation associated with the owner
associated with the migrating file system. may not be associated with the migrating file system.
When a source server propagates owner-related state associated with When a source server propagates owner-related state associated with
owners that span multiple file systems, it will propagate the owner owners that span multiple file systems, it will propagate the owner
sequence value to the destination server, while retaining it on the sequence value to the destination server, while retaining it on the
source server, as long as there exists state associated with the source server, as long as there exists state associated with the
owner. When owner information is propagated in this way, source and owner. When owner information is propagated in this way, source and
destination servers start with the same owner sequence value which is destination servers start with the same owner sequence value that is
then updated independently, as the client makes owner-related then updated independently, as the client makes owner-related
requests to the servers. Note that each server will have some period requests to the servers. Note that each server will have some period
in which the associated sequence value for an owner is identical to in which the associated sequence value for an owner is identical to
the one transferred as part of migration. At those times, when a the one transferred as part of migration. At those times, when a
server receives a request with a matching owner sequence value, it server receives a request with a matching owner sequence value, it
MUST NOT respond with the associated stored response if the MUST NOT respond with the associated stored response if the
associated file system is not, when the reissued request is received, associated file system is not, when the reissued request is received,
part of the set of file systems handled by that server. part of the set of file systems handled by that server.
One sort of case may require more complex handling. When multiple One sort of case may require more complex handling. When multiple
file system are migrated, in sequence, to a specific destination file systems are migrated, in sequence, to a specific destination
server, an owner may be migrated to a destination server, on which it server, an owner may be migrated to a destination server, on which it
was already present, leading to the issue of how the resident owner was already present, leading to the issue of how the resident owner
information and that being newly migrated are to be reconciled. information and that being newly migrated are to be reconciled.
If file system migration encounters a situation where owner If file system migration encounters a situation where owner
information needs to be merged, it MAY decline to transfer such information needs to be merged, it MAY decline to transfer such
state, even if it chooses to handle other cases in which locks for a state, even if it chooses to handle other cases in which locks for a
given owner are spread among multiple file systems. given owner are spread among multiple file systems.
As a way of understanding the situations which need to be addressed As a way of understanding the situations that need to be addressed
when owner information needs to be merged, consider the following when owner information needs to be merged, consider the following
scenario: scenario:
o There is client C and two servers X and Y. There are two o There is client C and two servers, X and Y. There are two
clientid4's designating C, which are referred to as CX and CY. clientid4s designating C, which are referred to as CX and CY.
o Initially server X supports file systems F1, F2, F3, and F4. o Initially, server X supports file systems F1, F2, F3, and F4.
These will be migrated, one-at-a-time, to server Y. These will be migrated, one at a time, to server Y.
o While these migrations are proceeding, the client makes locking o While these migrations are proceeding, the client makes locking
requests for file system F1 through F4 on behalf of owner O requests for file systems F1 through F4 on behalf of owner O
(either a lock-owner or an open-owner), with each request going to (either a lock-owner or an open-owner), with each request going to
X or Y depending on where the relevant file system is being X or Y depending on where the relevant file system is being
supported at the time the request is made. supported at the time the request is made.
o Once the first migration event occurs, client C will maintain two o Once the first migration event occurs, client C will maintain two
instances for owner O, one for each server. instances for owner O, one for each server.
o It is always possible that C may make a request of server X o It is always possible that C may make a request of server X
relating to owner O, and before receiving a response, find the relating to owner O, and before receiving a response, it finds the
target file system has moved to Y, and need to re-issue the target file system has moved to Y and needs to reissue the request
request to server Y. to server Y.
o At the same time, C may make a request of server Y relating to o At the same time, C may make a request of server Y relating to
owner O, and this too may encounter a lost-response situation. owner O, and this too may encounter a lost-response situation.
As a result of such merger situations, the server will need to As a result of such merger situations, the server will need to
provide support for dealing with retransmission of owner-sequenced provide support for dealing with retransmission of owner-sequenced
requests that diverges from the typical model in which there is requests that diverge from the typical model in which there is
support for retransmission of replies only for a request whose support for retransmission of replies only for a request whose
sequence value exactly matches the last one sent. In some sequence value exactly matches the last one sent. In some
situations, there may be two requests, each of which had the last situations, there may be two requests, each of which had the last
sequence when it was issued. As a result of migration and owner sequence when it was issued. As a result of migration and owner
merger, one of those will no longer be the last by sequence. merger, one of those will no longer be the last by sequence.
When servers do support such merger of owner information on the When servers do support such merger of owner information on the
destination server, the following rules are to be adhered to: destination server, the following rules are to be adhered to:
o When an owner sequence value is propagated to a destination server o When an owner sequence value is propagated to a destination server
skipping to change at page 34, line 32 skipping to change at page 36, line 20
can be deduced that the client in question has received the reply. can be deduced that the client in question has received the reply.
As a result of the operation of these rules, there are three ways in As a result of the operation of these rules, there are three ways in
which there can be more reply data than what is typically present, which there can be more reply data than what is typically present,
i.e., data for a single request per owner whose sequence is the last i.e., data for a single request per owner whose sequence is the last
one received, where the next sequence to be used is one beyond that. one received, where the next sequence to be used is one beyond that.
o When the owner sequence value for a migrating file system is o When the owner sequence value for a migrating file system is
greater than the corresponding value on the destination server, greater than the corresponding value on the destination server,
the last request for the owner in effect at the destination server the last request for the owner in effect at the destination server
needs to be retained, even though it is no longer one less the needs to be retained, even though it is no longer one less than
next sequence to be received. the next sequence to be received.
o When the owner sequence value for a migrating file system is less o When the owner sequence value for a migrating file system is less
than the corresponding value on the destination server the than the corresponding value on the destination server, the
sequence number for last request for the owner in effect on the sequence number for last request for the owner in effect on the
migrating file system needs to be retained, even though it is no migrating file system needs to be retained, even though it is no
longer one than less the next sequence to be received. longer than one less the next sequence to be received.
o When the owner sequence value for a migrating file system is equal o When the owner sequence value for a migrating file system is equal
to the corresponding value on the destination server, one has two to the corresponding value on the destination server, one has two
different "last" requests which both must be retained. The next different "last" requests that both must be retained. The next
sequence value to be used is one beyond the sequence value shared sequence value to be used is one beyond the sequence value shared
by these two requests. by these two requests.
Here are some guidelines as to when servers can drop such additional Here are some guidelines as to when servers can drop such additional
reply data which is created as part of owner information migration. reply data, which is created as part of owner information migration.
o The server SHOULD NOT drop this information simply because it o The server SHOULD NOT drop this information simply because it
receives a new sequence value for the owner in question, since receives a new sequence value for the owner in question, since
that request may have been issued before the client was aware of that request may have been issued before the client was aware of
the migration event. the migration event.
o The server SHOULD drop this information if it receives a new o The server SHOULD drop this information if it receives a new
sequence value for the owner in question and the request relates sequence value for the owner in question, and the request relates
to the same file system. to the same file system.
o The server SHOULD drop the part of this information that relates o The server SHOULD drop the part of this information that relates
to non-migrated file systems, if it receives a new sequence value to non-migrated file systems if it receives a new sequence value
for the owner in question and the request relates to a non- for the owner in question, and the request relates to a non-
migrated file system. migrated file system.
o The server MAY drop this information when it receives a new o The server MAY drop this information when it receives a new
sequence value for the owner in question a considerable period of sequence value for the owner in question for a considerable period
time (more than one or two lease periods) after the migration of time (more than one or two lease periods) after the migration
occurs. occurs.
6.1.2. Replication and State 6.1.2. Replication and State
Since client switch-over in the case of replication is not under Since client switch-over in the case of replication is not under
server control, the handling of state is different. In this case, server control, the handling of state is different. In this case,
leases, stateids and client IDs do not have validity across a leases, stateids, and client IDs do not have validity across a
transition from one server to another. The client must re-establish transition from one server to another. The client must re-establish
its locks on the new server. This can be compared to the re- its locks on the new server. This can be compared to the re-
establishment of locks by means of reclaim-type requests after a establishment of locks by means of reclaim-type requests after a
server reboot. The difference is that the server has no provision to server reboot. The difference is that the server has no provision to
distinguish requests reclaiming locks from those obtaining new locks distinguish requests reclaiming locks from those obtaining new locks
or to defer the latter. Thus, a client re-establishing a lock on the or to defer the latter. Thus, a client re-establishing a lock on the
new server (by means of a LOCK or OPEN request), may have the new server (by means of a LOCK or OPEN request) may have the requests
requests denied due to a conflicting lock. Since replication is denied due to a conflicting lock. Since replication is intended for
intended for read-only use of file systems, such denial of locks read-only use of file systems, such denial of locks should not pose
should not pose large difficulties in practice. When an attempt to large difficulties in practice. When an attempt to re-establish a
re-establish a lock on a new server is denied, the client should lock on a new server is denied, the client should treat the situation
treat the situation as if its original lock had been revoked. as if its original lock had been revoked.
6.1.3. Notification of Migrated Lease 6.1.3. Notification of Migrated Lease
A file system can be migrated to another server while a client that A file system can be migrated to another server while a client that
has state related to that file system is not actively submitting has state related to that file system is not actively submitting
requests to it. In this case, the migration is reported to the requests to it. In this case, the migration is reported to the
client during lease renewal. Lease renewal can occur either client during lease renewal. Lease renewal can occur either
explicitly via a RENEW operation, or implicitly when the client explicitly via a RENEW operation or implicitly when the client
performs a lease-renewing operation on another file system on that performs a lease-renewing operation on another file system on that
server. server.
In order for the client to schedule renewal of leases that may have In order for the client to schedule renewal of leases that may have
been relocated to the new server, the client must find out about been relocated to the new server, the client must find out about
lease relocation before those leases expire. Similarly, when lease relocation before those leases expire. Similarly, when
migration occurs but there has not been transparent state migration, migration occurs but there has not been transparent state migration,
the client needs to find out about the change soon enough to be able the client needs to find out about the change soon enough to be able
to reclaim the lock within the destination server's grace period. To to reclaim the lock within the destination server's grace period. To
accomplish this, all operations which implicitly renew leases for a accomplish this, all operations that implicitly renew leases for a
client (such as OPEN, CLOSE, READ, WRITE, RENEW, LOCK, and others), client (such as OPEN, CLOSE, READ, WRITE, RENEW, LOCK, and others)
will return the error NFS4ERR_LEASE_MOVED if responsibility for any will return the error NFS4ERR_LEASE_MOVED if responsibility for any
of the leases to be renewed has been transferred to a new server. of the leases to be renewed has been transferred to a new server.
Note that when the transfer of responsibility leaves remaining state Note that when the transfer of responsibility leaves remaining state
for that lease on the source server, the lease is renewed just as it for that lease on the source server, the lease is renewed just as it
would have been in the NFS4ERR_OK case, despite returning the error. would have been in the NFS4ERR_OK case, despite returning the error.
The transfer of responsibility happens when the server receives a The transfer of responsibility happens when the server receives a
GETATTR(fs_locations) from the client for each file system for which GETATTR(fs_locations) from the client for each file system for which
a lease has been moved to a new server. Normally it does this after a lease has been moved to a new server. Normally, it does this after
receiving an NFS4ERR_MOVED for an access to the file system but the receiving an NFS4ERR_MOVED for an access to the file system, but the
server is not required to verify that this happens in order to server is not required to verify that this happens in order to
terminate the return of NFS4ERR_LEASE_MOVED. By convention, the terminate the return of NFS4ERR_LEASE_MOVED. By convention, the
compounds containing GETATTR(fs_locations) SHOULD include an appended compounds containing GETATTR(fs_locations) SHOULD include an appended
RENEW operation to permit the server to identify the client getting RENEW operation to permit the server to identify the client getting
the information. the information.
Note that the NFS4ERR_LEASE_MOVED error is only required when Note that the NFS4ERR_LEASE_MOVED error is required only when
responsibility for at least one stateid has been affected. In the responsibility for at least one stateid has been affected. In the
case of a null lease, where the only associated state is a clientid4, case of a null lease, where the only associated state is a clientid4,
an NFS4ERR_LEASE_MOVED error SHOULD NOT be generated. an NFS4ERR_LEASE_MOVED error SHOULD NOT be generated.
Upon receiving the NFS4ERR_LEASE_MOVED error, a client that supports Upon receiving the NFS4ERR_LEASE_MOVED error, a client that supports
file system migration MUST perform the necessary GETATTR operation file system migration MUST perform the necessary GETATTR operation
for each of the file systems containing state that have been migrated for each of the file systems containing state that have been
and so give the server evidence that it is aware of the migration of migrated, so it gives the server evidence that it is aware of the
the file system. Once the client has done this for all migrated file migration of the file system. Once the client has done this for all
systems on which the client holds state, the server MUST resume migrated file systems on which the client holds state, the server
normal handling of stateful requests from that client. MUST resume normal handling of stateful requests from that client.
One way in which clients can do this efficiently in the presence of One way in which clients can do this efficiently in the presence of
large numbers of file systems is described below. This approach large numbers of file systems is described below. This approach
divides the process into two phases, one devoted to finding the divides the process into two phases: one devoted to finding the
migrated file systems and the second devoted to doing the necessary migrated file systems, and the second devoted to doing the necessary
GETATTRs. GETATTRs.
The client can find the migrated file systems by building and issuing The client can find the migrated file systems by building and issuing
one or more COMPOUND requests, each consisting of a set of PUTFH/ one or more COMPOUND requests, each consisting of a set of PUTFH/
GETFH pairs, each pair using a filehandle in one of the file systems GETFH pairs, each pair using a filehandle in one of the file systems
in question. All such COMPOUND requests can be done in parallel. in question. All such COMPOUND requests can be done in parallel.
The successful completion of such a request indicates that none of The successful completion of such a request indicates that none of
the file systems interrogated have been migrated while termination the file systems interrogated have been migrated while termination
with NFS4ERR_MOVED indicates that the file system getting the error with NFS4ERR_MOVED indicates that the file system getting the error
has migrated while those interrogated before it in the same COMPOUND has migrated while those interrogated before it in the same COMPOUND
have not. Those whose interrogation follows the error remain in an have not. Those whose interrogation follows the error remain in an
uncertain state and can be interrogated by restarting the requests uncertain state and can be interrogated by restarting the requests
from after the point at which NFS4ERR_MOVED was returned or by from after the point at which NFS4ERR_MOVED was returned or by
issuing a new set of COMPOUND requests for the file systems which issuing a new set of COMPOUND requests for the file systems that
remain in an uncertain state. remain in an uncertain state.
Once the migrated file systems have been found, all that is needed is Once the migrated file systems have been found, all that is needed is
for the client to give evidence to the server that it is aware of the for the client to give evidence to the server that it is aware of the
migrated status of file systems found by this process, by migrated status of file systems found by this process, by
interrogating the fs_locations attribute for a filehandle within each interrogating the fs_locations attribute for a filehandle within each
of the migrated file systems. The client can do this by building and of the migrated file systems. The client can do this by building and
issuing one or more COMPOUND requests, each of which consists of a issuing one or more COMPOUND requests, each of which consists of a
set of PUTFH operations, each followed by a GETATTR of the set of PUTFH operations, each followed by a GETATTR of the
fs_locations attribute. A RENEW is necessary to enable the fs_locations attribute. A RENEW is necessary to enable the
operations to be associated with the lease returning operations to be associated with the lease returning
NFS4ERR_LEASE_MOVED. Once the client has done this for all migrated NFS4ERR_LEASE_MOVED. Once the client has done this for all migrated
file systems on which the client holds state, the server will resume file systems on which the client holds state, the server will resume
normal handling of stateful requests from that client. normal handling of stateful requests from that client.
In order to support legacy clients that do not handle the In order to support legacy clients that do not handle the
NFS4ERR_LEASE_MOVED error correctly, the server SHOULD time out after NFS4ERR_LEASE_MOVED error correctly, the server SHOULD time out after
a wait of at least two lease periods, at which time it will resume a wait of at least two lease periods, at which time it will resume
normal handling of stateful requests from all clients. If a client normal handling of stateful requests from all clients. If a client
attempts to access the migrated files, the server MUST reply attempts to access the migrated files, the server MUST reply with
NFS4ERR_MOVED. In this situation, it is likely that the client would NFS4ERR_MOVED. In this situation, it is likely that the client would
find its lease expired although a server may use "courtesy" locks (as find its lease expired, although a server may use "courtesy" locks
described in Section 9.6.3.1 of [RFC7530]) to mitigate the issue. (as described in Section 9.6.3.1 of [RFC7530]) to mitigate the issue.
When the client receives an NFS4ERR_MOVED error, the client can When the client receives an NFS4ERR_MOVED error, the client can
follow the normal process to obtain the destination server follow the normal process to obtain the destination server
information (through the fs_locations attribute) and perform renewal information (through the fs_locations attribute) and perform renewal
of those leases on the new server. If the server has not had state of those leases on the new server. If the server has not had state
transferred to it transparently, the client will receive either transferred to it transparently, the client will receive either
NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new server, NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new server,
as described above. The client can then recover state information as as described above. The client can then recover state information as
it does in the event of server failure. it does in the event of server failure.
skipping to change at page 37, line 51 skipping to change at page 39, line 39
client may wish to retrieve fs_locations information from a server. client may wish to retrieve fs_locations information from a server.
When a server becomes unresponsive, for example, a client may use When a server becomes unresponsive, for example, a client may use
cached fs_locations data to discover an alternate server hosting the cached fs_locations data to discover an alternate server hosting the
same file system data. A client may periodically request same file system data. A client may periodically request
fs_locations data from a server in order to keep its cache of fs_locations data from a server in order to keep its cache of
fs_locations data fresh. fs_locations data fresh.
Since a GETATTR(fs_locations) operation would be used for refreshing Since a GETATTR(fs_locations) operation would be used for refreshing
cached fs_locations data, a server could mistake such a request as cached fs_locations data, a server could mistake such a request as
indicating recognition of an NFS4ERR_LEASE_MOVED condition. indicating recognition of an NFS4ERR_LEASE_MOVED condition.
Therefore a compound which is not intended to signal that a client Therefore, a compound that is not intended to signal that a client
has recognized a migrated lease SHOULD be prefixed with a guard has recognized a migrated lease SHOULD be prefixed with a guard
operation which fails with NFS4ERR_MOVED if the file handle being operation that fails with NFS4ERR_MOVED if the filehandle being
queried is no longer present on the server. The guard can be as queried is no longer present on the server. The guard can be as
simple as a GETFH operation. simple as a GETFH operation.
Though unlikely, it is possible that the target of such a compound Though unlikely, it is possible that the target of such a compound
could be migrated in the time after the guard operation is executed could be migrated in the time after the guard operation is executed
on the server but before the GETATTR(fs_locations) operation is on the server but before the GETATTR(fs_locations) operation is
encountered. When a client issues a GETATTR(fs_locations) operation encountered. When a client issues a GETATTR(fs_locations) operation
as part of a compound not intended to signal recognition of a as part of a compound not intended to signal recognition of a
migrated lease, it SHOULD be prepared to process fs_locations data in migrated lease, it SHOULD be prepared to process fs_locations data in
the reply that shows the current location of the file system is gone. the reply that shows the current location of the file system is gone.
6.1.4. Migration and the Lease_time Attribute 6.1.4. Migration and the lease_time Attribute
In order that the client may appropriately manage its leases in the In order that the client may appropriately manage its leases in the
case of migration, the destination server must establish proper case of migration, the destination server must establish proper
values for the lease_time attribute. values for the lease_time attribute.
When state is transferred transparently, that state should include When state is transferred transparently, that state should include
the correct value of the lease_time attribute. The lease_time the correct value of the lease_time attribute. The lease_time
attribute on the destination server must never be less than that on attribute on the destination server must never be less than that on
the source since this would result in premature expiration of leases the source since this would result in premature expiration of leases
granted by the source server. Upon migration in which state is granted by the source server. Upon migration in which state is
transferred transparently, the client is under no obligation to re- transferred transparently, the client is under no obligation to
fetch the lease_time attribute and may continue to use the value refetch the lease_time attribute and may continue to use the value
previously fetched (on the source server). previously fetched (on the source server).
In the case in which lease merger occurs as part of state transfer, In the case in which lease merger occurs as part of state transfer,
the lease_time attribute of the destination lease remains in effect. the lease_time attribute of the destination lease remains in effect.
The client can simply renew that lease with its existing lease_time The client can simply renew that lease with its existing lease_time
attribute. State in the source lease is renewed at the time of attribute. State in the source lease is renewed at the time of
transfer so that it cannot expire, as long as the destination lease transfer so that it cannot expire, as long as the destination lease
is appropriately renewed. is appropriately renewed.
If state has not been transferred transparently (i.e., the client If state has not been transferred transparently (i.e., the client
needs to reclaim or re-obtain its locks), the client should fetch the needs to reclaim or re-obtain its locks), the client should fetch the
value of lease_time on the new (i.e., destination) server, and use it value of lease_time on the new (i.e., destination) server, and use it
for subsequent locking requests. However the server must respect a for subsequent locking requests. However, the server must respect a
grace period at least as long as the lease_time on the source server, grace period at least as long as the lease_time on the source server,
in order to ensure that clients have ample time to reclaim their in order to ensure that clients have ample time to reclaim their
locks before potentially conflicting non-reclaimed locks are granted. locks before potentially conflicting non-reclaimed locks are granted.
The means by which the new server obtains the value of lease_time on The means by which the new server obtains the value of lease_time on
the old server is left to the server implementations. It is not the old server is left to the server implementations. It is not
specified by the NFS version 4.0 protocol. specified by the NFS version 4.0 protocol.
7. Server Implementation Considerations 7. Server Implementation Considerations
This chapter provides suggestions to help server implementers deal This section provides suggestions to help server implementers deal
with issues involved in the transparent transfer of file system- with issues involved in the transparent transfer of file-system-
related data between servers. Servers are not obliged to follow related data between servers. Servers are not obliged to follow
these suggestions, but should be sure that their approach to the these suggestions but should be sure that their approach to the
issues handle all the potential problems addressed below. issues handle all the potential problems addressed below.
7.1. Relation of Locking State Transfer to Other Aspects of File System 7.1. Relation of Locking State Transfer to Other Aspects of File System
Motion Motion
In many cases, state transfer will be part of a larger function In many cases, state transfer will be part of a larger function
wherein the contents of a file system are transferred from server to wherein the contents of a file system are transferred from server to
server. Although specifics will vary with the implementation, the server. Although specifics will vary with the implementation, the
relation between the transfer of persistent file data and metadata relation between the transfer of persistent file data and metadata
and the transfer of state will typically be described by one of the and the transfer of state will typically be described by one of the
skipping to change at page 39, line 36 skipping to change at page 41, line 20
accessible from multiple servers, and transferring the right and accessible from multiple servers, and transferring the right and
responsibility for handling that file system from server to responsibility for handling that file system from server to
server. server.
In such implementations, the transfer of locking state happens on In such implementations, the transfer of locking state happens on
its own, as described in Section 7.2. The transfer of physical its own, as described in Section 7.2. The transfer of physical
access to the file system happens after the locking state is access to the file system happens after the locking state is
transferred and before any subsequent access to the file system. transferred and before any subsequent access to the file system.
In cases where such transfer is not instantaneous, there will be a In cases where such transfer is not instantaneous, there will be a
period in which all operations on the file system are held off, period in which all operations on the file system are held off,
either by having the operations themselves return NFS4ERR_DELAY, either by having the operations themselves return NFS4ERR_DELAY
or, where this is not allowed, by using the techniques described or, where this is not allowed, by using the techniques described
below in Section 7.2. below in Section 7.2.
o In other implementations, file system data and metadata must be o In other implementations, file system data and metadata must be
copied from the server where it has existed to the destination copied from the server where they have existed to the destination
server. Because of the typical amounts of data involved, it is server. Because of the typical amounts of data involved, it is
generally not practical to hold off access to the file system generally not practical to hold off access to the file system
while this transfer is going on. Normal access to the file while this transfer is going on. Normal access to the file
system, including modifying operations, will generally happen system, including modifying operations, will generally happen
while the transfer is going on. while the transfer is going on.
Eventually the file system copying process will complete. At this Eventually, the file system copying process will complete. At
point, there will be two valid copies of the file system, one on this point, there will be two valid copies of the file system, one
each of the source and destination servers. Servers may maintain on each of the source and destination servers. Servers may
that state of affairs by making sure that each modification to maintain that state of affairs by making sure that each
file system data is done on both the source and destination modification to file system data is done on both the source and
servers. destination servers.
Although the transfer of locking state can begin before the above Although the transfer of locking state can begin before the above
state of affairs is reached, servers will often wait until it is state of affairs is reached, servers will often wait until it is
arrived at to begin transfer of locking state. Once the transfer arrived at to begin transfer of locking state. Once the transfer
of locking state is completed, as described in the section below, of locking state is completed, as described in the section below,
clients may be notified of the migration event and access the clients may be notified of the migration event and access the
destination file system on the destination server. destination file system on the destination server.
o Another case in which file system data and metadata must be copied o Another case in which file system data and metadata must be copied
from server to server involves a variant of the pattern above. In from server to server involves a variant of the pattern above. In
cases in which a single file system moves between or among a small cases in which a single file system moves between or among a small
set of servers, it will transition to a server on which a previous set of servers, it will transition to a server on which a previous
instantiation of that same file system existed before. In such instantiation of that same file system existed before. In such
cases, it is often more efficient to update the previous file cases, it is often more efficient to update the previous file
system instance to reflect changes made while the active file system instance to reflect changes made while the active file
system was residing elsewhere, rather than copying the file system system was residing elsewhere rather than copying the file system
data anew. data anew.
In such cases, the copying of file system data and metadata is In such cases, the copying of file system data and metadata is
replaced by a process which validates each visible file system replaced by a process that validates each visible file system
object, copying new objects and updating those that have changed object, copying new objects and updating those that have changed
since the file system was last present on the destination server. since the file system was last present on the destination server.
Although this process is generally shorter than a complete copy, Although this process is generally shorter than a complete copy,
it is generally long enough that it is not practical to hold off it is generally long enough that it is not practical to hold off
access to the file system while this update is going on. access to the file system while this update is going on.
Eventually the file system updating process will complete. At Eventually, the file system updating process will complete. At
this point, there will be two valid copies of the file system, one this point, there will be two valid copies of the file system, one
on each of the source and destination servers. Servers may on each of the source and destination servers. Servers may
maintain that state of affairs just as is done in the previous maintain that state of affairs just as is done in the previous
case. Similarly, the transfer of locking state, once it is case. Similarly, the transfer of locking state, once it is
complete, allows the clients to be notified of the migration event complete, allows the clients to be notified of the migration event
and access the destination file system on the destination server. and access the destination file system on the destination server.
7.2. Preventing Locking State Modification During Transfer 7.2. Preventing Locking State Modification during Transfer
When transferring locking state from the source to a destination When transferring locking state from the source to a destination
server, there will be occasions when the source server will need to server, there will be occasions when the source server will need to
prevent operations that modify the state being transferred. For prevent operations that modify the state being transferred. For
example, if the locking state at time T is sent to the destination example, if the locking state at time T is sent to the destination
server, any state change that occurs on the source server after that server, any state change that occurs on the source server after that
time but before the file system transfer is made effective will mean time but before the file system transfer is made effective will mean
that the state on the destination server will differ from that on the that the state on the destination server will differ from that on the
source server, which matches what the client would expect to see. source server, which matches what the client would expect to see.
In general, a server can prevent some set of server-maintained data In general, a server can prevent some set of server-maintained data
from changing by returning NFS4ERR_DELAY on operations which attempt from changing by returning NFS4ERR_DELAY on operations that attempt
to change that data. In the case of locking state for NFSv4.0, there to change that data. In the case of locking state for NFSv4.0, there
are two specific issues that might interfere: are two specific issues that might interfere:
o Returning NFS4ERR_DELAY will not prevent state from changing in o Returning NFS4ERR_DELAY will not prevent state from changing in
that owner-based sequence values will still change, even though that owner-based sequence values will still change, even though
NFS4ERR_DELAY is returned. For example OPEN and LOCK will change NFS4ERR_DELAY is returned. For example, OPEN and LOCK will change
state (in the form of owner seqid values) even when they return state (in the form of owner seqid values) even when they return
NFS4ERR_DELAY. NFS4ERR_DELAY.
o Some operations which modify locking state are not allowed to o Some operations that modify locking state are not allowed to
return NFS4ERR_DELAY (i.e., OPEN_CONFIRM, RELEASE_LOCKOWNER, and return NFS4ERR_DELAY (i.e., OPEN_CONFIRM, RELEASE_LOCKOWNER, and
RENEW) RENEW).
Note that the first problem and most instances of the second can be Note that the first problem and most instances of the second can be
addressed by returning NFS4ERR_DELAY on the operations that establish addressed by returning NFS4ERR_DELAY on the operations that establish
a filehandle within the target as one of the filehandles associated a filehandle within the target as one of the filehandles associated
with the request, i.e., as either the current or saved filehandle. with the request, i.e., as either the current or saved filehandle.
This would require returning NFS4ERR_DELAY under the following This would require returning NFS4ERR_DELAY under the following
circumstances: circumstances:
o On a PUTFH that specifies a filehandle within the target file o On a PUTFH that specifies a filehandle within the target file
system. system.
o On a LOOKUP or LOOKUPP that crosses into the target file system. o On a LOOKUP or LOOKUPP that crosses into the target file system.
As a result of doing this, OPEN_CONFIRM is dealt with, leaving only As a result of doing this, OPEN_CONFIRM is dealt with, leaving only
RELEASE_LOCKOWNER and RENEW still to be dealt with. RELEASE_LOCKOWNER and RENEW still to be dealt with.
Note that if the server establishes and maintains a situation in Note that if the server establishes and maintains a situation in
which no request has, as either the current or saved filehandle, a which no request has, as either the current or saved filehandle, a
filehandle within the target file system, no special handling of filehandle within the target file system, no special handling of
SAVEFH or RESTOREFH is required. Thus the fact that these operations SAVEFH or RESTOREFH is required. Thus, the fact that these
cannot return NFS4ERR_DELAY is not a problem since neither will operations cannot return NFS4ERR_DELAY is not a problem since neither
establish a filehandle in the target file system as the current will establish a filehandle in the target file system as the current
filehandle. filehandle.
If the server is to establish the situation described above, it may If the server is to establish the situation described above, it may
have to take special note of long-running requests which started have to take special note of long-running requests that started
before state migration. Part of any solution to this issue will before state migration. Part of any solution to this issue will
involve distinguishing two separate points in time at which handling involve distinguishing two separate points in time at which handling
for the target file system will change. Let us distinguish; for the target file system will change. Let us distinguish:
o A time T after which the previously mentioned operations will o A time T after which the previously mentioned operations will
return NFS4ERR_DELAY. return NFS4ERR_DELAY.
o A later time T' at which the server can consider file system o A later time T' at which the server can consider file system
locking state fixed, making it possible for it to be sent to the locking state fixed, making it possible for it to be sent to the
destination server. destination server.
For a server to decide on T', it must ensure that requests started For a server to decide on T', it must ensure that requests started
before T, cannot change target file system locking state, given that before T cannot change target file system locking state, given that
all those started after T are dealt with by returning NFS4ERR_DELAY all those started after T are dealt with by returning NFS4ERR_DELAY
upon setting filehandles within the target file system. Among the upon setting filehandles within the target file system. Among the
ways of doing this are: ways of doing this are:
o Keeping track of the earliest request started which is still in o Keeping track of the earliest request started that is still in
execution (for example, by keeping a list of active requests execution (for example, by keeping a list of active requests
ordered by request start time). Requests that started before and ordered by request start time). Requests that started before and
are still in progress at time T may potentially affect the locking are still in progress at time T may potentially affect the locking
state; once the starting time of the earliest-started active state; once the starting time of the earliest-started active
request is later than T, the starting time of the first such request is later than T, the starting time of the first such
request can be chosen as T' by the server since any request in request can be chosen as T' by the server since any request in
progress after T' started after time T. Accordingly it would not progress after T' started after time T. Accordingly, it would not
have been allowed to change locking state for the migrating file have been allowed to change locking state for the migrating file
system and would have returned NFS4ERR_DELAY had it tried to make system and would have returned NFS4ERR_DELAY had it tried to make
a change. a change.
o Keeping track of the count of requests, started before time T o Keeping track of the count of requests started before time T that
which have a filehandle within the target file system as either have a filehandle within the target file system as either the
the current or saved filehandle. The server can then define T' to current or saved filehandle. The server can then define T' to be
be the first time after T at which the count is zero. the first time after T at which the count is zero.
The set of operations that change locking state include two that The set of operations that change locking state include two that
cannot be dealt with by the above approach, because they are not cannot be dealt with by the above approach, because they are not
specific to a particular file system and do not use a current specific to a particular file system and do not use a current
filehandle as an implicit parameter. filehandle as an implicit parameter.
o RENEW can be dealt with by applying the renewal to state for non- o RENEW can be dealt with by applying the renewal to state for non-
transitioning file systems. The effect of renewal for the transitioning file systems. The effect of renewal for the
transitioning file system can be ignored, as long as the servers transitioning file system can be ignored, as long as the servers
make sure that the lease on the destination server has an make sure that the lease on the destination server has an
expiration time that is no earlier than the latest renewal done on expiration time that is no earlier than the latest renewal done on
the source server. This can be easily accomplished by making the the source server. This can be easily accomplished by making the
lease expiration on the destination server equal to the time the lease expiration on the destination server equal to the time in
state transfer was completed plus the lease period. which the state transfer was completed plus the lease period.
o RELEASE_LOCKOWNER can be handled by propagating the fact of the o RELEASE_LOCKOWNER can be handled by propagating the fact of the
lock-owner deletion (e.g., by using an RPC) to the destination lock-owner deletion (e.g., by using an RPC) to the destination
server. Such a propagation RPC can be done as part of the server. Such a propagation RPC can be done as part of the
operation or the existence of the deletion can be recorded locally operation, or the existence of the deletion can be recorded
and propagation of owner deletions to the destination server done locally and propagation of owner deletions to the destination
as a batch later. In either case, the actual deletions on the server done as a batch later. In either case, the actual
destination server have to be delayed until all of the other state deletions on the destination server have to be delayed until all
information has been transferred. of the other state information has been transferred.
Alternatively, RELEASE_LOCKOWNER can be dealt with by returning Alternatively, RELEASE_LOCKOWNER can be dealt with by returning
NFS4ERR_DELAY. In order to avoid compatibility issues for clients NFS4ERR_DELAY. In order to avoid compatibility issues for clients
not prepared to accept NFS4ERR_DELAY in response to not prepared to accept NFS4ERR_DELAY in response to
RELEASE_LOCKOWNER, care must be exercised. (See Section 8.3 for RELEASE_LOCKOWNER, care must be exercised. (See Section 8.3 for
details.) details.)
The approach outlined above, wherein NFS$ERR_DELAY is returned based The approach outlined above, wherein NFS4ERR_DELAY is returned based
primarily on the use of current and saved filehandles in the file primarily on the use of current and saved filehandles in the file
system, prevents all reference to the transitioning file system, system, prevents all reference to the transitioning file system
rather than limiting the delayed operations to those that change rather than limiting the delayed operations to those that change
locking state on the transitioning file system. Because of this, locking state on the transitioning file system. Because of this,
servers may choose to limit the time during which this broad approach servers may choose to limit the time during which this broad approach
is used by adopting a layered approach to the issue. is used by adopting a layered approach to the issue.
o During the preparatory phase, operations that change, create, or o During the preparatory phase, operations that change, create, or
destroy locks or modify the valid set of stateids will return destroy locks or modify the valid set of stateids will return
NFS4ERR_DELAY. During this phase, owner-associated seqids may NFS4ERR_DELAY. During this phase, owner-associated seqids may
change, and the identity of the file system associated with the change, and the identity of the file system associated with the
last request for a given owner may change as well. Also, last request for a given owner may change as well. Also,
skipping to change at page 43, line 43 skipping to change at page 45, line 27
system either the current or saved filehandle for a request. system either the current or saved filehandle for a request.
RELEASE_LOCKOWNER operations may return NFS4ERR_DELAY, but if they RELEASE_LOCKOWNER operations may return NFS4ERR_DELAY, but if they
are processed, the lock-owner deletion needs to be communicated are processed, the lock-owner deletion needs to be communicated
immediately to the destination server. immediately to the destination server.
A possible sequence would be the following. A possible sequence would be the following.
o The server enters the preparatory phase for the transitioning file o The server enters the preparatory phase for the transitioning file
system. system.
o At this point locking state, including stateids, locks, owner o At this point, locking state, including stateids, locks, and owner
strings are transferred to the destination server. The seqids strings, is transferred to the destination server. The seqids
associated with owners are either not transferred, or transferred associated with owners are either not transferred or transferred
on a provisional basis, subject to later change. on a provisional basis, subject to later change.
o After the above has been transferred, the server may enter the o After the above has been transferred, the server may enter the
restrictive phase for the file system. restrictive phase for the file system.
o At this point, the updated seqid values may be sent to the o At this point, the updated seqid values may be sent to the
destination server. destination server.
Reporting regarding pending owner deletions (as a result of Reporting regarding pending owner deletions (as a result of
RELEASE_LOCKOWNER operations) can be communicated at the same RELEASE_LOCKOWNER operations) can be communicated at the same
time. time.
o Once it is known that all of this information has been transferred o Once it is known that all of this information has been transferred
to the destination server, and there are no pending to the destination server, and there are no pending
RELEASE_LOCKOWNER notifications outstanding, the source server may RELEASE_LOCKOWNER notifications outstanding, the source server may
treat the file system transition as having occurred and return treat the file system transition as having occurred and return
NFS4ERR_MOVED when an attempt is made to access it. NFS4ERR_MOVED when an attempt is made to access it.
8. Additional Changes 8. Additional Changes
This chapter contains a number of items which relate to the changes This section contains a number of items that relate to the changes in
in the chapters above, but which, for one reason or another, exist in the section above, but which, for one reason or another, exist in
different portions of the specification to be updated. different portions of the specification to be updated.
8.1. Summary of Additional Changes from Previous Documents 8.1. Summary of Additional Changes from Previous Documents
Summarized here are all the remaining changes, not included in the Summarized here are all the remaining changes, not included in the
two main chapters. two main sections.
o New definition of the error NFS4ERR_CLID_INUSE, appearing in o New definition of the error NFS4ERR_CLID_INUSE, appearing in
Section 8.2. This replaces the definition in Section 13.1.10.1 in Section 8.2. This replaces the definition in Section 13.1.10.1 in
[RFC7530]. [RFC7530].
o A revision of the error definitions chapter to allow o A revision of the error definitions section to allow
RELEASE_LOCKOWNER to return NFS4ERR_DELAY, with appropriate RELEASE_LOCKOWNER to return NFS4ERR_DELAY, with appropriate
constraints to assure interoperability with clients not expecting constraints to assure interoperability with clients not expecting
this error to be returned. These changes are discussed in this error to be returned. These changes are discussed in
Section 8.2 and modify the error tables in Sections 13.2 and 13.4 Section 8.2 and modify the error tables in Sections 13.2 and 13.4
in [RFC7530]. in [RFC7530].
o A revised description of SETCLIENTID, appearing in Section 8.4. o A revised description of SETCLIENTID, appearing in Section 8.4.
This brings the description into sync with the rest of the spec This brings the description into sync with the rest of the
regarding NFS4ERR_CLID_INUSE. The revised description replace the specification regarding NFS4ERR_CLID_INUSE. The revised
one in Section 16.33 of [RFC7530]. description replaces the one in Section 16.33 of [RFC7530].
o Some security-related changes appear in Sections 8.5 and 8.6. The o Some security-related changes appear in Sections 8.5 and 8.6. The
Security Considerations section of this document (Section 9) Security Considerations section of this document (Section 9)
describes the effect on the corresponding section (Section 19) in describes the effect on the corresponding section (Section 19) in
[RFC7530]. [RFC7530].
8.2. NFS4ERR_CLID_INUSE definition 8.2. NFS4ERR_CLID_INUSE Definition
The definition of this error is now as follows The definition of this error is now as follows:
The SETCLIENTID operation has found that the id string within the The SETCLIENTID operation has found that the id string within the
specified nfs_client_id4 was previously presented with a different specified nfs_client_id4 was previously presented with a different
principal and that client instance currently holds an active principal and that client instance currently holds an active
lease. A server MAY return this error if the same principal is lease. A server MAY return this error if the same principal is
used but a change in authentication flavor gives good reason to used, but a change in authentication flavor gives good reason to
reject the new SETCLIENTID operation as not bona fide. reject the new SETCLIENTID operation as not bona fide.
8.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER 8.3. NFS4ERR_DELAY Return from RELEASE_LOCKOWNER
The existing error tables should be considered modified to allow The existing error tables should be considered modified to allow
NFS4ERR_DELAY to be returned by RELEASE_LOCKOWNER. However, the NFS4ERR_DELAY to be returned by RELEASE_LOCKOWNER. However, the
scope of this addition is limited and is not to be considered as scope of this addition is limited and is not to be considered as
making this error return generally acceptable. making this error return generally acceptable.
It needs to be made clear that servers may not return this error to It needs to be made clear that servers may not return this error to
clients not prepared to support file system migration. Such clients clients not prepared to support file system migration. Such clients
may be following the error specifications in [RFC7530] and so might may be following the error specifications in [RFC7530] and so might
not expect NFS4ERR_DELAY to be returned on RELEASE_LOCKOWNER. not expect NFS4ERR_DELAY to be returned on RELEASE_LOCKOWNER.
skipping to change at page 45, line 39 skipping to change at page 47, line 16
if it were a note appearing together with the newly allowed error if it were a note appearing together with the newly allowed error
code: code:
In order to make server state fixed for a file system being In order to make server state fixed for a file system being
migrated, a server MAY return NFS4ERR_DELAY in response to a migrated, a server MAY return NFS4ERR_DELAY in response to a
RELEASE_LOCKOWNER that will affect locking state being propagated RELEASE_LOCKOWNER that will affect locking state being propagated
to a destination server. The source server MUST NOT do so unless to a destination server. The source server MUST NOT do so unless
it is likely that it will later return NFS4ERR_MOVED for the file it is likely that it will later return NFS4ERR_MOVED for the file
system in question. system in question.
In the context of lock-owner release, the set of file systems such In the context of lock-owner release, the set of file systems,
that server state being made fixed can result in NFS4ERR_DELAY such that server state being made fixed can result in
must include the file system on which the operation associated NFS4ERR_DELAY, must include the file system on which the operation
with the current lock-owner seqid was performed. associated with the current lock-owner seqid was performed.
In addition, this set may include other file systems on which an In addition, this set may include other file systems on which an
operation associated with an earlier seqid for the current lock- operation associated with an earlier seqid for the current lock-
owner seqid was performed, since servers will have to deal with owner seqid was performed, since servers will have to deal with
the issue of an owner being used in succession for multiple file the issue of an owner being used in succession for multiple file
systems. systems.
Thus, if a client that is prepared to receive NFS4ERR_MOVED after Thus, if a client is prepared to receive NFS4ERR_MOVED after
creating state associated with a given file system, it also needs creating state associated with a given file system, it also needs
to be prepared to receive NFS4ERR_DELAY in response to to be prepared to receive NFS4ERR_DELAY in response to
RELEASE_LOCKOWNER, if it has used that owner in connection with a RELEASE_LOCKOWNER, if it has used that owner in connection with a
file on that file system. file on that file system.
8.4. Operation 35: SETCLIENTID - Negotiate Client ID 8.4. Operation 35: SETCLIENTID -- Negotiate Client ID
8.4.1. SYNOPSIS 8.4.1. SYNOPSIS
client, callback, callback_ident -> clientid, setclientid_confirm client, callback, callback_ident -> clientid, setclientid_confirm
8.4.2. ARGUMENT 8.4.2. ARGUMENT
struct SETCLIENTID4args { struct SETCLIENTID4args {
nfs_client_id4 client; nfs_client_id4 client;
cb_client4 callback; cb_client4 callback;
skipping to change at page 46, line 43 skipping to change at page 48, line 27
default: default:
void; void;
}; };
8.4.4. DESCRIPTION 8.4.4. DESCRIPTION
The client uses the SETCLIENTID operation to notify the server of its The client uses the SETCLIENTID operation to notify the server of its
intention to use a particular client identifier, callback, and intention to use a particular client identifier, callback, and
callback_ident for subsequent requests that entail creating lock, callback_ident for subsequent requests that entail creating lock,
share reservation, and delegation state on the server. Upon share reservation, and delegation state on the server. Upon
successful completion the server will return a shorthand client ID successful completion, the server will return a shorthand client ID
which, if confirmed via a separate step, will be used in subsequent that, if confirmed via a separate step, will be used in subsequent
file locking and file open requests. Confirmation of the client ID file locking and file open requests. Confirmation of the client ID
must be done via the SETCLIENTID_CONFIRM operation to return the must be done via the SETCLIENTID_CONFIRM operation to return the
client ID and setclientid_confirm values, as verifiers, to the client ID and setclientid_confirm values, as verifiers, to the
server. The reason why two verifiers are necessary is that it is server. The reason why two verifiers are necessary is that it is
possible to use SETCLIENTID and SETCLIENTID_CONFIRM to modify the possible to use SETCLIENTID and SETCLIENTID_CONFIRM to modify the
callback and callback_ident information but not the shorthand client callback and callback_ident information but not the shorthand client
ID. In that event, the setclientid_confirm value is effectively the ID. In that event, the setclientid_confirm value is effectively the
only verifier. only verifier.
The callback information provided in this operation will be used if The callback information provided in this operation will be used if
skipping to change at page 47, line 33 skipping to change at page 49, line 21
x be the value of the client.id subfield of the SETCLIENTID4args x be the value of the client.id subfield of the SETCLIENTID4args
structure. structure.
v be the value of the client.verifier subfield of the v be the value of the client.verifier subfield of the
SETCLIENTID4args structure. SETCLIENTID4args structure.
c be the value of the client ID field returned in the c be the value of the client ID field returned in the
SETCLIENTID4resok structure. SETCLIENTID4resok structure.
k represent the value combination of the fields callback and k represent the value combination of the callback and callback_ident
callback_ident fields of the SETCLIENTID4args structure. fields of the SETCLIENTID4args structure.
s be the setclientid_confirm value returned in the SETCLIENTID4resok s be the setclientid_confirm value returned in the SETCLIENTID4resok
structure. structure.
{ v, x, c, k, s } be a quintuple for a client record. A client { v, x, c, k, s } be a quintuple for a client record. A client
record is confirmed if there has been a SETCLIENTID_CONFIRM record is confirmed if there has been a SETCLIENTID_CONFIRM
operation to confirm it. Otherwise it is unconfirmed. An operation to confirm it. Otherwise, it is unconfirmed. An
unconfirmed record is established by a SETCLIENTID call. unconfirmed record is established by a SETCLIENTID call.
8.4.5.1. IMPLEMENTATION (Preparatory Phase) 8.4.5.1. IMPLEMENTATION (Preparatory Phase)
Since SETCLIENTID is a non-idempotent operation, our treatment Since SETCLIENTID is a non-idempotent operation, our treatment
assumes use of a duplicate request cache (DRC). For a discussion of assumes use of a duplicate request cache (DRC). For a discussion of
the DRC, see Section 9.1.7 of [RFC7530]. the DRC, see Section 9.1.7 of [RFC7530].
When the server gets a SETCLIENTID { v, x, k } request, it first does When the server gets a SETCLIENTID { v, x, k } request, it first does
a number of preliminary checks as listed below before proceeding to a number of preliminary checks as listed below before proceeding to
the main part of SETCLIENTID processing. the main part of SETCLIENTID processing.
o It first looks up the request in the DRC. If there is a hit, it o It first looks up the request in the DRC. If there is a hit, it
returns the result cached in the DRC. The server does NOT remove returns the result cached in the DRC. The server does NOT remove
client state (locks, shares, delegations) nor does it modify any client state (locks, shares, delegations) nor does it modify any
recorded callback and callback_ident information for client { x }. recorded callback and callback_ident information for client { x }.
The server now proceeds to the main part of SETCLIENTID. The server now proceeds to the main part of SETCLIENTID.
o Otherwise (i.e., in the case of any DRC miss), the server takes o Otherwise (i.e., in the case of any DRC miss), the server takes
the client id string x, and searches for confirmed client records the client ID string x and searches for confirmed client records
for x that the server may have recorded from previous SETCLIENTID for x that the server may have recorded from previous SETCLIENTID
calls. If there are no such, or if all such records have a calls. If there are no such records, or if all such records have
recorded principal which matches that of the current request's a recorded principal that matches that of the current request's
principal, then the preparatory phase proceeds as follows. principal, then the preparatory phase proceeds as follows.
* If there is a confirmed client record with a matching client id * If there is a confirmed client record with a matching client ID
string and a non-matching principal, the server checks the string and a non-matching principal, the server checks the
current state of the associated lease. If there is no current state of the associated lease. If there is no
associated state for the lease, or the lease has expired, the associated state for the lease, or the lease has expired, the
server proceeds to the main part of SETCLIENTID. server proceeds to the main part of SETCLIENTID.
* Otherwise, the server is being asked to do a SETCLIENTID for a * Otherwise, the server is being asked to do a SETCLIENTID for a
client by a non-matching principal while there is active state. client by a non-matching principal while there is active state.
In this case, the server rejects the SETCLIENTID request In this case, the server rejects the SETCLIENTID request
returning an NFS4ERR_CLID_INUSE error, since use of a single returning an NFS4ERR_CLID_INUSE error, since use of a single
client with multiple principals is not allowed. Note that even client with multiple principals is not allowed. Note that even
though the previously used clientaddr4 is returned with this though the previously used clientaddr4 is returned with this
error, the use of the same id string with multiple error, the use of the same id string with multiple clientaddr4s
clientaddr4's is not prohibited, while its use with multiple is not prohibited, while its use with multiple principals is
principals is prohibited. prohibited.
8.4.5.2. IMPLEMENTATION (Main Phase) 8.4.5.2. IMPLEMENTATION (Main Phase)
If the SETCLIENTID has not been dealt with by DRC processing, and has If the SETCLIENTID has not been dealt with by DRC processing, and has
not been rejected with an NFS4ERR_CLID_INUSE error, then the main not been rejected with an NFS4ERR_CLID_INUSE error, then the main
part of SETCLIENTID processing proceeds, as described below. part of SETCLIENTID processing proceeds, as described below.
o The server checks if it has recorded a confirmed record for { v, o The server checks if it has recorded a confirmed record for { v,
x, c, l, s }, where l may or may not equal k. If so, and since x, c, l, s }, where l may or may not equal k. If so, and since
the id verifier v of the request matches that which is confirmed the id verifier v of the request matches that which is confirmed
and recorded, the server treats this as a probable callback and recorded, the server treats this as a probable callback
information update and records an unconfirmed { v, x, c, k, t } information update and records an unconfirmed { v, x, c, k, t }
and leaves the confirmed { v, x, c, l, s } in place, such that t and leaves the confirmed { v, x, c, l, s } in place, such that t
!= s. It does not matter if k equals l or not. Any pre-existing != s. It does not matter if k equals l or not. Any pre-existing
unconfirmed { v, x, c, *, * } is removed. unconfirmed { v, x, c, *, * } is removed.
The server returns { c, t }. It is indeed returning the old The server returns { c, t }. It is indeed returning the old
clientid4 value c, because the client apparently only wants to clientid4 value c, because the client apparently only wants to
update callback value k to value l. It's possible this request is update callback value k to value l. It's possible this request is
one from the Byzantine router that has stale callback information, one from the Byzantine router that has stale callback information,
but this is not a problem. The callback information update is but this is not a problem. The callback information update is
only confirmed if followed up by a SETCLIENTID_CONFIRM { c, t }. only confirmed if followed up by a SETCLIENTID_CONFIRM { c, t }.
The server awaits confirmation of k via SETCLIENTID_CONFIRM { c, t The server awaits confirmation of k via SETCLIENTID_CONFIRM { c, t
}. }.
The server does NOT remove client (lock/share/delegation) state The server does NOT remove client (lock/share/delegation) state
skipping to change at page 50, line 26 skipping to change at page 52, line 14
8.5. Security Considerations for Inter-server Information Transfer 8.5. Security Considerations for Inter-server Information Transfer
Although the means by which the source and destination server Although the means by which the source and destination server
communicate is not specified by NFSv4.0, the following security- communicate is not specified by NFSv4.0, the following security-
related considerations for inter-server communication should be related considerations for inter-server communication should be
noted. noted.
o Communication between source and destination servers needs to be o Communication between source and destination servers needs to be
carried out in a secure manner, with protection against deliberate carried out in a secure manner, with protection against deliberate
modification of data in transit provided either by use of private modification of data in transit provided by using either a private
network, or by using a security mechanism that ensures integrity. network or a security mechanism that ensures integrity. In many
In many cases, privacy will also be required, requiring a cases, privacy will also be required, requiring a strengthened
strengthened security mechanism if a private network is not used. security mechanism if a private network is not used.
o Effective implementation of the file system migration function o Effective implementation of the file system migration function
requires that a trust relationship exist between source and requires that a trust relationship exist between source and
destination servers. The details of that trust relationship destination servers. The details of that trust relationship
depend on the specifics of the inter-server transfer protocol, depend on the specifics of the inter-server transfer protocol,
which is outside the scope of this specification. which is outside the scope of this specification.
o The source server may communicate to the destination server o The source server may communicate to the destination server
security-related information in order to allow it to more security-related information in order to allow it to more
rigorously validate clients' identity. For example, the rigorously validate clients' identity. For example, the
skipping to change at page 51, line 7 skipping to change at page 52, line 42
operation to be performed by the client that would not be allowed operation to be performed by the client that would not be allowed
otherwise. otherwise.
8.6. Security Considerations Revision 8.6. Security Considerations Revision
The penultimate paragraph of Section 19 of [RFC7530] should be The penultimate paragraph of Section 19 of [RFC7530] should be
revised to read as follows: revised to read as follows:
Because the operations SETCLIENTID/SETCLIENTID_CONFIRM are Because the operations SETCLIENTID/SETCLIENTID_CONFIRM are
responsible for the release of client state, it is imperative that responsible for the release of client state, it is imperative that
the principal used for these operations is checked against and the principal used for these operations be checked against and
match the previous use of these operations. In addition, use of match the previous use of these operations. In addition, use of
integrity protection is desirable on the SETCLIENTID operation, to integrity protection is desirable on the SETCLIENTID operation, to
prevent an attack whereby a change in the boot instance id prevent an attack whereby a change in the boot instance id
(verifier) forces an undesired loss of client state. See the (verifier) forces an undesired loss of client state. See
Section 5 for further discussion. Section 5 for further discussion.
9. Security Considerations 9. Security Considerations
The security considerations of [RFC7530] remain appropriate with the The security considerations of [RFC7530] remain appropriate with the
exception of the modification to the penultimate paragraph specified exception of the modification to the penultimate paragraph specified
in Section 8.6 of this document and the addition of the material in in Section 8.6 of this document and the addition of the material in
Section 8.5. Section 8.5.
10. IANA Considerations 10. References
This document does not require actions by IANA.
11. References
11.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
[RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System
(NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530,
March 2015, <http://www.rfc-editor.org/info/rfc7530>. March 2015, <http://www.rfc-editor.org/info/rfc7530>.
11.2. Informative References 10.2. Informative References
[info-migr] [INFO-MIGR]
Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker, Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker,
"NFSv4 migration: Implementation experience and spec "NFSv4 migration: Implementation experience and spec
issues to resolve", March 2015, <http://www.ietf.org/id/ issues to resolve", Work in Progress, draft-ietf-nfsv4-
draft-ietf-nfsv4-migration-issues-07.txt>. migration-issues-09, February 2016.
Work in progress.
[RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS [RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS
Version 3 Protocol Specification", RFC 1813, Version 3 Protocol Specification", RFC 1813,
DOI 10.17487/RFC1813, June 1995, DOI 10.17487/RFC1813, June 1995,
<http://www.rfc-editor.org/info/rfc1813>. <http://www.rfc-editor.org/info/rfc1813>.
[RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed.,
"Network File System (NFS) Version 4 Minor Version 1 "Network File System (NFS) Version 4 Minor Version 1
Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010,
<http://www.rfc-editor.org/info/rfc5661>. <http://www.rfc-editor.org/info/rfc5661>.
Appendix A. Acknowledgements Acknowledgements
The editor and authors of this document gratefully acknowledge the The editor and authors of this document gratefully acknowledge the
contributions of Trond Myklebust of Primary Data and Robert Thurlow contributions of Trond Myklebust of Primary Data and Robert Thurlow
of Oracle. We also thank Tom Haynes of Primary Data and Spencer of Oracle. We also thank Tom Haynes of Primary Data and Spencer
Shepler of Microsoft for their guidance and suggestions. Shepler of Microsoft for their guidance and suggestions.
Special thanks go to members of the Oracle Solaris NFS team, Special thanks go to members of the Oracle Solaris NFS team,
especially Rick Mesta and James Wahlig, for their work implementing especially Rick Mesta and James Wahlig, for their work implementing
an NFSv4.0 migration prototype and identifying many of the issues an NFSv4.0 migration prototype and identifying many of the issues
addressed here. addressed here.
Authors' Addresses Authors' Addresses
David Noveck (editor) David Noveck (editor)
Hewlett Packard Enterprise Hewlett Packard Enterprise
165 Dascomb Road 165 Dascomb Road
Andover, MA 01810 Andover, MA 01810
US United States of America
Phone: +1 978 474 2011 Phone: +1 978 474 2011
Email: davenoveck@gmail.com Email: davenoveck@gmail.com
Piyush Shivam Piyush Shivam
Oracle Corporation Oracle Corporation
5300 Riata Park Ct. 5300 Riata Park Ct.
Austin, TX 78727 Austin, TX 78727
US United States of America
Phone: +1 512 401 1019 Phone: +1 512 401 1019
Email: piyush.shivam@oracle.com Email: piyush.shivam@oracle.com
Charles Lever Charles Lever
Oracle Corporation Oracle Corporation
1015 Granger Avenue 1015 Granger Avenue
Ann Arbor, MI 48104 Ann Arbor, MI 48104
US United States of America
Phone: +1 734 274 2396 Phone: +1 734 274 2396
Email: chuck.lever@oracle.com Email: chuck.lever@oracle.com
Bill Baker Bill Baker
Oracle Corporation Oracle Corporation
5300 Riata Park Ct. 5300 Riata Park Ct.
Austin, TX 78727 Austin, TX 78727
US United States of America
Phone: +1 512 401 1081 Phone: +1 512 401 1081
Email: bill.baker@oracle.com Email: bill.baker@oracle.com
 End of changes. 269 change blocks. 
538 lines changed or deleted 535 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/