draft-ietf-nfsv4-rfc3530-migration-update-02.txt   draft-ietf-nfsv4-rfc3530-migration-update-03.txt 
NFSv4 D. Noveck, Ed. NFSv4 D. Noveck, Ed.
Internet-Draft EMC Internet-Draft EMC
Updates: 3530 (if approved) P. Shivam Updates: 3530 (if approved) P. Shivam
Intended status: Standards Track C. Lever Intended status: Standards Track C. Lever
Expires: February 09, 2014 B. Baker Expires: April 15, 2014 B. Baker
ORACLE ORACLE
August 08, 2013 October 12, 2013
NFSv4.0 migration: Specification Update NFSv4.0 migration: Specification Update
draft-ietf-nfsv4-rfc3530-migration-update-02 draft-ietf-nfsv4-rfc3530-migration-update-03
Abstract Abstract
The migration feature of NFSv4 allows for responsibility for a single The migration feature of NFSv4 allows for responsibility for a single
filesystem to move from one server to another, without disruption to filesystem to move from one server to another, without disruption to
clients. Recent implementation experience has shown problems in the clients. Recent implementation experience has shown problems in the
existing specification for this feature in NFSv4.0. This document existing specification for this feature in NFSv4.0. This document
clarifies and corrects the NFSv4.0 specification (RFC3530 and clarifies and corrects the NFSv4.0 specification (RFC3530 and
possible successors) to address these problems. possible successors) to address these problems.
skipping to change at page 1, line 38 skipping to change at page 1, line 38
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 09, 2014. This Internet-Draft will expire on April 15, 2014.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 30 skipping to change at page 2, line 30
4.6. Uniform client id string Approach . . . . . . . . . . . . 13 4.6. Uniform client id string Approach . . . . . . . . . . . . 13
4.7. Mixing client id string Approaches . . . . . . . . . . . 14 4.7. Mixing client id string Approaches . . . . . . . . . . . 14
4.8. Trunking Determination when using Uniform client id 4.8. Trunking Determination when using Uniform client id
strings . . . . . . . . . . . . . . . . . . . . . . . . . 16 strings . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.9. Client id string construction details . . . . . . . . . . 21 4.9. Client id string construction details . . . . . . . . . . 21
5. Locking and Multi-Server Namespace . . . . . . . . . . . . . 22 5. Locking and Multi-Server Namespace . . . . . . . . . . . . . 22
5.1. Changes from Replaced Sections . . . . . . . . . . . . . 22 5.1. Changes from Replaced Sections . . . . . . . . . . . . . 22
5.2. Lock State and Filesystem Transitions . . . . . . . . . . 23 5.2. Lock State and Filesystem Transitions . . . . . . . . . . 23
5.3. Migration and State . . . . . . . . . . . . . . . . . . . 23 5.3. Migration and State . . . . . . . . . . . . . . . . . . . 23
5.3.1. Migration and clientid's . . . . . . . . . . . . . . 24 5.3.1. Migration and clientid's . . . . . . . . . . . . . . 24
5.3.2. Migration and state owner information . . . . . . . . 25 5.3.2. Migration and state owner information . . . . . . . . 26
5.4. Replication and State . . . . . . . . . . . . . . . . . . 29 5.4. Replication and State . . . . . . . . . . . . . . . . . . 29
5.5. Notification of Migrated Lease . . . . . . . . . . . . . 29 5.5. Notification of Migrated Lease . . . . . . . . . . . . . 30
5.6. Migration and the Lease_time Attribute . . . . . . . . . 32 5.6. Migration and the Lease_time Attribute . . . . . . . . . 33
6. Server Implementation Considerations . . . . . . . . . . . . 32 6. Server Implementation Considerations . . . . . . . . . . . . 33
6.1. Relation of Locking State Transfer to Other Aspects of 6.1. Relation of Locking State Transfer to Other Aspects of
Filesystem Motion . . . . . . . . . . . . . . . . . . . . 32 Filesystem Motion . . . . . . . . . . . . . . . . . . . . 33
6.2. Preventing Locking State Modification During Transfer . . 34 6.2. Preventing Locking State Modification During Transfer . . 35
7. Additional Changes . . . . . . . . . . . . . . . . . . . . . 37 7. Additional Changes . . . . . . . . . . . . . . . . . . . . . 38
7.1. Summary of Additional Changes from Previous Documents . . 37 7.1. Summary of Additional Changes from Previous Documents . . 38
7.2. NFS4ERR_CLID_INUSE definition . . . . . . . . . . . . . . 37 7.2. NFS4ERR_CLID_INUSE definition . . . . . . . . . . . . . . 39
7.3. Operation 35: SETCLIENTID - Negotiate Client ID . . . . . 38 7.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER . . . . . . . 39
7.4. Security Considerations revision . . . . . . . . . . . . 42 7.4. Operation 35: SETCLIENTID - Negotiate Client ID . . . . . 40
8. Security Considerations . . . . . . . . . . . . . . . . . . . 42 7.5. Security Considerations revision . . . . . . . . . . . . 44
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 42 8. Security Considerations . . . . . . . . . . . . . . . . . . . 44
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 42 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 43 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 44
11.1. Normative References . . . . . . . . . . . . . . . . . . 43 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 45
11.2. Informative References . . . . . . . . . . . . . . . . . 43 11.1. Normative References . . . . . . . . . . . . . . . . . . 45
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 11.2. Informative References . . . . . . . . . . . . . . . . . 45
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 45
1. Introduction 1. Introduction
This document is a standards track document which corrects the This document is a standards track document which corrects the
existing definitive specification of the NFSv4.0 protocol, in existing definitive specification of the NFSv4.0 protocol, in
[RFC3530] and the one expected to become definitive (now in [RFC3530] and the one expected to become definitive (now in
[cur-rfc3530-bis]). Given this fact, one should take the current [cur-rfc3530-bis]). Given this fact, one should take the current
document into account when learning about NFSv4.0, particularly if document into account when learning about NFSv4.0, particularly if
one is concerned with issues that relate to: one is concerned with issues that relate to:
skipping to change at page 6, line 34 skipping to change at page 6, line 34
struct nfs_client_id4 { struct nfs_client_id4 {
verifier4 verifier; verifier4 verifier;
opaque id<NFS4_OPAQUE_LIMIT>; opaque id<NFS4_OPAQUE_LIMIT>;
}; };
The nfs_client_id4 structure uniquely defines a client boot instance The nfs_client_id4 structure uniquely defines a client boot instance
as follows: as follows:
o The id field is a variable-length string which uniquely identifies o The id field is a variable-length string which uniquely identifies
a specific client. Although, we describe it as a string and it is a specific client. Although, we describe it as a string and it is
often referred to a "client string," it should be understood that often referred to as a "client string," it should be understood
the protocol defines this as opaque data. In particular, those that the protocol defines this as opaque data. In particular,
receiving such an id should not assume that it will be in the those receiving such an id should not assume that it will be in
UTF-8 encoding. Servers MUST NOT reject an nfs_client_id4 simply the UTF-8 encoding. Servers MUST NOT reject an nfs_client_id4
because the id string does not follow the rules of UTF-8 encoding. simply because the id string does not follow the rules of UTF-8
encoding.
The string MAY be different for each server network address that The string MAY be different for each server network address that
the client accesses, rather than common to all server network the client accesses, rather than common to all server network
addresses. addresses.
o The verifier is a client incarnation identifier that is used by o The verifier is a client incarnation identifier that is used by
the server to detect client reboots. Only if the verifier is the server to detect client reboots. Only if the verifier is
different from that which the server has previously recorded in different from that which the server has previously recorded in
connection with the client (as identified by the id field) does connection with the client (as identified by the id field) does
the server cancel the client's leased state, once it receives the server cancel the client's leased state, once it receives
skipping to change at page 8, line 46 skipping to change at page 8, line 46
string. For each client, the set of distinct owner values used with string. For each client, the set of distinct owner values used with
that client constitutes the set of owners of that type, for the given that client constitutes the set of owners of that type, for the given
client. client.
Each open is associated with a specific open-owner while each byte- Each open is associated with a specific open-owner while each byte-
range lock is associated with a lock-owner and an open-owner, the range lock is associated with a lock-owner and an open-owner, the
latter being the open-owner associated with the open file under which latter being the open-owner associated with the open file under which
the LOCK operation was done. the LOCK operation was done.
When a clientid4 is presented to a server and that clientid4 is not When a clientid4 is presented to a server and that clientid4 is not
recognized, the server will reject the request with the error valid, the server will reject the request with the an error that
NFS4ERR_STALE_CLIENTID. This can occur for a number of reasons: depends on the reason for clientid4 invalidity. The error
NFS4ERR_ADMIN_REVOKED is returned when the invalidation is the result
of administrative action, When the clientid4 is unrecognizable, the
error NFS4ERR_STALE_CLIENTID or NFS4ERR_EXPIRED may be returned. An
unrecognizable clientid4 can occur for a number of reasons:
o A server reboot causing loss of the server's knowledge of the o A server reboot causing loss of the server's knowledge of the
client client. (Always returns NFS4ERR_STALE_CLIENTID)
o Client error sending an incorrect clientid4 or a valid clientid4 o Client error sending an incorrect clientid4 or a valid clientid4
to the wrong server. to the wrong server. (May return either error).
o Loss of lease state due to lease expiration. o Loss of lease state due to lease expiration. (Always returns
NFS4ERR_EXPIRED)
o Client or server error causing the server to believe that the o Client or server error causing the server to believe that the
client has rebooted (i.e. receiving a SETCLIENTID with an client has rebooted (i.e. receiving a SETCLIENTID with an
nfs_client_id4 which has a matching id string and a non-matching nfs_client_id4 which has a matching id string and a non-matching
boot verifier). boot verifier). (May return either error).
o Migration of all state under the associated lease causes its non- o Migration of all state under the associated lease causes its non-
existence to be recognized on the source server. existence to be recognized on the source server. (Always returns
NFS4ERR_STALE_CLIENTID)
o Merger of state under the associated lease with another lease o Merger of state under the associated lease with another lease
under a different clientid causes the clientid4 serving as the under a different clientid causes the clientid4 serving as the
source of the merge to cease being recognized on its server. source of the merge to cease being recognized on its server.
(Always returns NFS4ERR_STALE_CLIENTID)
In the event of a server reboot, or loss of lease state due to lease In the event of a server reboot, loss of lease state due to lease
expiration, the client must obtain a new clientid4 by use of the expiration, or administrative revocation of a clientid4, the client
SETCLIENTID operation and then proceed to any other necessary must obtain a new clientid4 by use of the SETCLIENTID operation and
recovery for the server reboot case (See the section entitled "Server then proceed to any other necessary recovery for the server reboot
Failure and Recovery"). In cases of server or client error resulting case (See the section entitled "Server Failure and Recovery"). In
in this error, use of SETCLIENTID to establish a new lease is cases of server or client error resulting in this error, use of
desirable as well. SETCLIENTID to establish a new lease is desirable as well.
In the last two cases, different recovery procedures are required. In the last two cases, different recovery procedures are required.
See Section 5.3 for details. Note that in cases in which there is See Section 5.3 for details. Note that in cases in which there is
any uncertainty about which sort of handling is applicable, the any uncertainty about which sort of handling is applicable, the
distinguishing characteristic is that in reboot-like cases, the distinguishing characteristic is that in reboot-like cases, the
clientid4 and all associated stateids cease to exist while in clientid4 and all associated stateids cease to exist while in
migration-related cases, the clientid4 ceases to exist while the migration-related cases, the clientid4 ceases to exist while the
stateids are still valid. stateids are still valid.
The client must also employ the SETCLIENTID operation when it The client must also employ the SETCLIENTID operation when it
skipping to change at page 23, line 35 skipping to change at page 23, line 35
5.3. Migration and State 5.3. Migration and State
In the case of migration, the servers involved in the migration of a In the case of migration, the servers involved in the migration of a
filesystem SHOULD transfer all server state associated with the filesystem SHOULD transfer all server state associated with the
migrating filesystem from source to the destination server. This migrating filesystem from source to the destination server. This
must be done in a way that is transparent to the client. This state must be done in a way that is transparent to the client. This state
transfer will ease the client's transition when a filesystem transfer will ease the client's transition when a filesystem
migration occurs. If the servers are successful in transferring all migration occurs. If the servers are successful in transferring all
state, the client will continue to use stateids assigned by the state, the client will continue to use stateids assigned by the
original server. Therefore the new server must recognize these original server. Therefore the new server must recognize these
stateids as valid. stateids as valid and treat them as representing the same locks as
they did on the source server.
In this context, the phrase "the same locks" means:
o That they are associated with the same file
o That they represent the same types of locks, whether opens,
delegations, advisory byte-range locks, or mandatory byte-range
locks.
o That they have the same lock particulars, including such things as
access modes, deny modes, and byte ranges.
o That they are associated with the same owner string(s).
If transferring stateids from server to server would result in a If transferring stateids from server to server would result in a
conflict for an existing stateid for the destination server with the conflict for an existing stateid for the destination server with the
existing client, transparent state migration MUST NOT happen for that existing client, transparent state migration MUST NOT happen for that
client. Servers participating in using transparent state migration client. Servers participating in using transparent state migration
should co-ordinate their stateid assignment policies to make this should co-ordinate their stateid assignment policies to make this
situation unlikely or impossible. The means by which this might be situation unlikely or impossible. The means by which this might be
done, like all of the inter-server interactions for migration, are done, like all of the inter-server interactions for migration, are
not specified by the NFS version 4.0 protocol. not specified by the NFS version 4.0 protocol.
skipping to change at page 24, line 13 skipping to change at page 24, line 26
the state was purged due to boot verifier mismatch. the state was purged due to boot verifier mismatch.
o If the stateid is valid, transparent state migration has occurred. o If the stateid is valid, transparent state migration has occurred.
Since responsibility for an entire filesystem is transferred with a Since responsibility for an entire filesystem is transferred with a
migration event, there is no possibility that conflicts will arise on migration event, there is no possibility that conflicts will arise on
the destination server as a result of the transfer of locks. the destination server as a result of the transfer of locks.
The servers may choose not to transfer the state information upon The servers may choose not to transfer the state information upon
migration. However, this choice is discouraged, except where migration. However, this choice is discouraged, except where
specific issues such as stateid conflicts make it necessary. In the specific issues such as stateid conflicts make it necessary. When a
case of migration without state transfer, when the client presents server implements migration and it does not transfer state
state information from the original server (e.g. in a RENEW op or a information, it SHOULD provide a filesystem-specific grace period, to
READ op of zero length), the client must be prepared to receive allow clients to reclaim locks associated with files in the migrated
either NFS4ERR_STALE_CLIENTID or NFS4ERR_STALE_STATEID from the new filesystem. If it did not do so, clients would have to re-obtain
server. The client should then recover its state information as it locks, with no assurance that a conflicting lock was not granted
normally would in response to a server failure. The new server must after the filesystem was migrated and before the lock was re-
take care to allow for the recovery of state information as it would obtained.
in the event of server restart.
In the case of migration without state transfer, when the client
presents state information from the original server (e.g. in a RENEW
op or a READ op of zero length), the client must be prepared to
receive either NFS4ERR_STALE_CLIENTID or NFS4ERR_BAD_STATEID from the
new server. The client should then recover its state information as
it normally would in response to a server failure. The new server
must take care to allow for the recovery of state information as it
would in the event of server restart.
In those situations in which state has not been transferred, as shown In those situations in which state has not been transferred, as shown
by a return of NFS4ERR_BAD_STATEID, the client may attempt to reclaim by a return of NFS4ERR_BAD_STATEID, the client may attempt to reclaim
locks in order to take advantage of cases in which the destination locks in order to take advantage of cases in which the destination
server has set up a file-system-specific grace period in support of server has set up a file-system-specific grace period in support of
the migration. the migration.
5.3.1. Migration and clientid's 5.3.1. Migration and clientid's
Handling of clientid values is similar to that for stateids. Handling of clientid values is similar to that for stateids.
However, there are some differences that derive from the fact that a However, there are some differences that derive from the fact that a
clientid is an object which spans multiple filesystems while a clientid is an object which spans multiple filesystems while a
stateid is inherently limited to a single filesystem. stateid is inherently limited to a single filesystem.
The clientid4 and nfs_client_id4 information (id string and boot The clientid4 and nfs_client_id4 information (id string and boot
verifier) will be transferred with the rest of the state information verifier) will be transferred with the rest of the state information
and the destination server should use that information to determine and the destination server should use that information to determine
appropriate clientid4 handling. Although the destination server may appropriate clientid4 handling. Although the destination server may
make state stored under an existing lease available under the make state stored under an existing lease available under the
skipping to change at page 25, line 52 skipping to change at page 26, line 24
This information includes: This information includes:
o The sequence number of the last operation associated with the o The sequence number of the last operation associated with the
particular owner. particular owner.
o Information regarding the results of the last operation, o Information regarding the results of the last operation,
sufficient to allow reissued operations to be correctly responded sufficient to allow reissued operations to be correctly responded
to. to.
When clients are implemented to isolate each openowner and lockowner When clients are implemented to isolate each openowner and lockowner
to a particular filesystem, the server may transfer this information to a particular filesystem, the server SHOULD transfer this
together with the lock state. The owner ceases to exist on the information together with the lock state. The owner ceases to exist
source server and is reconstituted on the destination server. on the source server and is reconstituted on the destination server.
Note that when servers take this approach for all owners whose state Note that when servers take this approach for all owners whose state
is limited to the particular filesystem being migrated, doing so will is limited to the particular filesystem being migrated, doing so will
not cause difficulties for clients not adhering to an approach in not cause difficulties for clients not adhering to an approach in
which owners are isolated to particular filesystems. As long as the which owners are isolated to particular filesystems. As long as the
client recognizes the loss of transferred state, the protocol allows client recognizes the loss of transferred state, the protocol allows
the owner in question to disappear and the client may have to deal the owner in question to disappear and the client may have to deal
with an owner confirmation request. with an owner confirmation request that would not have occurred in
the absence of the migration.
When migration occurs and the source server discovers an owner whose When migration occurs and the source server discovers an owner whose
state includes the migrated filesystem but other filesystems as well, state includes the migrated filesystem but other filesystems as well,
it MAY decline to transfer the associated state. In this case, use it cannot transfer the associated owner state. Instead, the existing
of the associated stateids on the destination server will result in owner state stays in place but propagation of owner state is done as
NFS4ERR_BAD_STATEID while their use on the source server will result specified below
in NFS4ERR_STALE_STATEID. Also, the owner will remain on the source
server and that server needs to ensure that a request reissue
associated with the migrated filesystem is not inappropriately acted
on. For example, a reissued OPEN for a file on the migrated
filesystem should not return a stateid that has been made invalid on
the server returning it, as a consequence of the migration event.
The source server MAY choose to migrate state associated with owners o When the current seqid for an owner represents an operation
that span multiple filesystems. In such cases, it needs to propagate associated with the filesystem being migrated, owner status SHOULD
the owner sequence value to the destination server, while retaining be propagated to the destination filesystem.
it on the source server, as long as there exists state associated
with the owner. When owner information is propagated in this way, o When the current seqid for an owner does not represent an
source and destination servers start with the same owner sequence operation associated with the filesystem being migrated, owner
value which is then updated independently, as the client makes owner- status MAY be propagated to the destination filesystem.
related requests to the servers. Note that each server will have
some period in which the associated sequence value for an owner is o When the owner in question has never been used for an operation
identical to the one transferred as part of migration. At those involving the migrated filesystem, the owner information SHOULD
times, when a server receives a request with a matching owner NOT be propagated to the destination filesystem.
sequence value, it MUST NOT respond with the associated stored
response if the associated filesystem is not, when the reissued Note that a server may obey all of the conditions above without the
request is received, part of the set of filesystems handled by that overhead of keeping track of set of filesystems that any particular
server. owner has been associated with. Consider a situation in which the
source server has decided to keep lock-related state associated with
a filesystem fixed, preparatory to propagating it to the destination
filesystem. If a client is free to create new locks associated with
existing owners on other filesystems, the owner information may be
propagated to the destination filesystem, even though, at the time
the filesystem migration is recognized by the client to have
occurred, the last operation associated with the owner may not be
associated with the migrating filesystem.
When source server propagates owner-related state associated with
owners that span multiple filesystems, it will propagate the owner
sequence value to the destination server, while retaining it on the
source server, as long as there exists state associated with the
owner. When owner information is propagated in this way, source and
destination servers start with the same owner sequence value which is
then updated independently, as the client makes owner-related
requests to the servers. Note that each server will have some period
in which the associated sequence value for an owner is identical to
the one transferred as part of migration. At those times, when a
server receives a request with a matching owner sequence value, it
MUST NOT respond with the associated stored response if the
associated filesystem is not, when the reissued request is received,
part of the set of filesystems handled by that server.
One sort of case may require more complex handling. When multiple One sort of case may require more complex handling. When multiple
filesystem are migrated, in sequence, to a specific destination filesystem are migrated, in sequence, to a specific destination
server, an owner may be migrated to a destination server, on which it server, an owner may be migrated to a destination server, on which it
was already present, leading to the issue of how the resident owner was already present, leading to the issue of how the resident owner
information and that being newly migrated are to be reconciled. information and that being newly migrated are to be reconciled.
If filesystem migration encounters a situation where owner If filesystem migration encounters a situation where owner
information needs to be merged, it MAY decline to transfer such information needs to be merged, it MAY decline to transfer such
state, even if it chooses to handle other cases in which locks for a state, even if it chooses to handle other cases in which locks for a
skipping to change at page 33, line 15 skipping to change at page 34, line 15
cases below. cases below.
o In some implementations, access to the on-disk contents of a o In some implementations, access to the on-disk contents of a
filesystem can be transferred from server to server by making the filesystem can be transferred from server to server by making the
storage devices on which the filesystem resides physically storage devices on which the filesystem resides physically
accessible from multiple servers, and transferring the right and accessible from multiple servers, and transferring the right and
responsibility for handling that filesystem from server to server. responsibility for handling that filesystem from server to server.
In such implementations, the transfer of locking state happens on In such implementations, the transfer of locking state happens on
its own, as described in Section 6.2. The transfer of physical its own, as described in Section 6.2. The transfer of physical
access to the file system happens after the locking state is access to the filesystem happens after the locking state is
transferred and before any subsequent access to the filesystem. transferred and before any subsequent access to the filesystem.
In cases where such transfer is not instantaneous, there will be a In cases where such transfer is not instantaneous, there will be a
period in which all operations on the filesystem are held off, period in which all operations on the filesystem are held off,
either by having the operations themselves return NFS4ERR_DELAY, either by having the operations themselves return NFS4ERR_DELAY,
or, where this is not allowed, by using the techniques described or, where this is not allowed, by using the techniques described
below in Section 6.2. below in Section 6.2.
o In other implementations, file system data and metadata must be o In other implementations, filesystem data and metadata must be
copied from the server where it has existed to the destination copied from the server where it has existed to the destination
server. Because of the typical amounts of data involved, it is server. Because of the typical amounts of data involved, it is
generally not practical to hold off access to the filesystem while generally not practical to hold off access to the filesystem while
this transfer is going on. Normal access to the filesystem, this transfer is going on. Normal access to the filesystem,
including modifying operations, will generally happen while the including modifying operations, will generally happen while the
transfer is going on. transfer is going on.
Eventually the filesystem copying process will complete. At this Eventually the filesystem copying process will complete. At this
point, there will be two valid copies of the filesystem, one on point, there will be two valid copies of the filesystem, one on
each of the source and destination servers. Servers may maintain each of the source and destination servers. Servers may maintain
skipping to change at page 33, line 45 skipping to change at page 34, line 45
filesystem data is done on both the source and destination filesystem data is done on both the source and destination
servers. servers.
Although the transfer of locking state can begin before the above Although the transfer of locking state can begin before the above
state of affairs is reached, servers will often wait until it is state of affairs is reached, servers will often wait until it is
arrived at to begin transfer of locking state. Once the transfer arrived at to begin transfer of locking state. Once the transfer
of locking state is completed, as described in the section below, of locking state is completed, as described in the section below,
clients may be notified of the migration event and access the clients may be notified of the migration event and access the
destination filesystem on the destination server. destination filesystem on the destination server.
o Another case in which file system data and metadata must be copied o Another case in which filesystem data and metadata must be copied
from server to server involves a variant of the pattern above. In from server to server involves a variant of the pattern above. In
cases in which a single filesystem moves between or among a small cases in which a single filesystem moves between or among a small
set of servers, it will transition to a server on which a previous set of servers, it will transition to a server on which a previous
instantiation of that same filesystem existed before. In such instantiation of that same filesystem existed before. In such
cases, it is often more efficient to update the previous cases, it is often more efficient to update the previous
filesystem instance to reflect changes made while the active filesystem instance to reflect changes made while the active
filesystem was residing elsewhere, rather than copying the filesystem was residing elsewhere, rather than copying the
filesystem data anew. filesystem data anew.
In such cases, the copying of file system data and metadata is In such cases, the copying of filesystem data and metadata is
replaced by a process which validates each visible filesystem replaced by a process which validates each visible filesystem
object, copying new objects and updating those that have changed object, copying new objects and updating those that have changed
since the filesystem was last present on the destination server. since the filesystem was last present on the destination server.
Although this process is generally shorter than a complete copy, Although this process is generally shorter than a complete copy,
it is generally long enough that it is not practical to hold off it is generally long enough that it is not practical to hold off
access to the filesystem while this update is going on. access to the filesystem while this update is going on.
Eventually the filesystem updating process will complete. At this Eventually the filesystem updating process will complete. At this
point, there will be two valid copies of the filesystem, one on point, there will be two valid copies of the filesystem, one on
each of the source and destination servers. Servers may maintain each of the source and destination servers. Servers may maintain
skipping to change at page 36, line 25 skipping to change at page 37, line 25
o RELEASE_LOCKOWNER can be handled by propagating the fact of the o RELEASE_LOCKOWNER can be handled by propagating the fact of the
lockowner deletion (e.g. by using an RPC) to the destination lockowner deletion (e.g. by using an RPC) to the destination
server. Such a propagation RPC can be done as part of the server. Such a propagation RPC can be done as part of the
operation or the existence of the deletion can be recorded locally operation or the existence of the deletion can be recorded locally
and propagation of owner deletions to the destination server done and propagation of owner deletions to the destination server done
as a batch later. In either case, the actual deletions on the as a batch later. In either case, the actual deletions on the
destination server have to be delayed until all of the other state destination server have to be delayed until all of the other state
information has been transferred. information has been transferred.
The approach outlined above prevents all reference to the Alternatively, RELEASE_LOCKOWNER can be dealt with by returning
transitioning filesystem, rather than limiting the delayed operations NFS4ERR_DELAY. In order to avoid compatibility issues for clients
to those that change locking state on the transitioning filesystem. not prepared to accept NFS4ERR_DELAY in response to
Because of this, servers may choose to limit the time during which RELEASE_LOCKOWNER, care must be exercised. (See Section 7.3 for
this broad approach is used by adopting a layered approach to the details.)
issue.
The approach outlined above, wherein NFS$ERR_DELAY is returned based
primarily on the use of current and saved filehandles in the
filesystem, prevents all reference to the transitioning filesystem,
rather than limiting the delayed operations to those that change
locking state on the transitioning filesystem. Because of this,
servers may choose to limit the time during which this broad approach
is used by adopting a layered approach to the issue.
o During the preparatory phase, operations that change, create, or o During the preparatory phase, operations that change, create, or
destroy locks or modify the valid set of stateid's will return destroy locks or modify the valid set of stateids will return
NFS4ERR_DELAY. During this phase, seqids may change. Also, NFS4ERR_DELAY. During this phase, owner-associated seqids may
RELEASE_LOCKOWNER operations may be processed as long as the fact change, and the identity of the filesystem associated with the
of the owner deletion is recorded locally for later transmission. last request for a given owner may change as well. Also,
RELEASE_LOCKOWNER operations may be processed without returning
NFS4ERR_DELAY as long as the fact of the lockowner deletion is
recorded locally for later transmission.
o During the restrictive phase, operations that change locking state o During the restrictive phase, operations that change locking state
for the filesystem in transition are prevented by returning for the filesystem in transition are prevented by returning
NFS4ERR_DELAY on any attempt to make a filehandle within the that NFS4ERR_DELAY on any attempt to make a filehandle within that
filesystem either the current or saved filehandle for a request. filesystem either the current or saved filehandle for a request.
RELEASE_LOCKOWNER operations may be processed only if the deletion
is communicated immediately to the destination server. RELEASE_LOCKOWNER operations may return NFS4ERR_DELAY, but if they
are processed, the lockowner deletion needs to be communicated
immediately to the destination server.
A possible sequence would be the following. A possible sequence would be the following.
o The server enters the preparatory phase for the transitioning o The server enters the preparatory phase for the transitioning
filesystem. filesystem.
o At this point locking state, including stateids, locks, owner o At this point locking state, including stateids, locks, owner
strings are transferred to the destination server. The seqids strings are transferred to the destination server. The seqids
associated with owners are either not transferred, or transferred associated with owners are either not transferred, or transferred
on a provisional basis, subject to later change. on a provisional basis, subject to later change.
skipping to change at page 37, line 42 skipping to change at page 39, line 5
main chapters. main chapters.
o New definition of the CLID_INUSE error. o New definition of the CLID_INUSE error.
o A revised description of SETCLIENTID, which brings the description o A revised description of SETCLIENTID, which brings the description
into sync with the rest of the spec regarding CLID_INUSE. into sync with the rest of the spec regarding CLID_INUSE.
o A revision to the Security Considerations section, indicating why o A revision to the Security Considerations section, indicating why
integrity protection is needed for the SETCLIENTID operation. integrity protection is needed for the SETCLIENTID operation.
o A revision of the error definitions chapter to allow
RELEASE_LOCKOWNER to return NFS4ERR_DELAY, with appropriate
constraints to assure interoperability with clients not expecting
this error to be returned.
7.2. NFS4ERR_CLID_INUSE definition 7.2. NFS4ERR_CLID_INUSE definition
The definition of this error is now as follows The definition of this error is now as follows
The SETCLIENTID operation has found that the id string within the The SETCLIENTID operation has found that the id string within the
specified nfs_client_id4 was previously presented with a different specified nfs_client_id4 was previously presented with a different
principal and that client instance currently holds an active principal and that client instance currently holds an active
lease. A server MAY return this error if the same principal is lease. A server MAY return this error if the same principal is
used but a change in authentication flavor gives good reason to used but a change in authentication flavor gives good reason to
reject the new SETCLIENTID operation as not bona fide. reject the new SETCLIENTID operation as not bona fide.
7.3. Operation 35: SETCLIENTID - Negotiate Client ID 7.3. NFS4ERR_DELAY return from RELEASE_LOCKOWNER
7.3.1. SYNOPSIS The existing error tables should be considered modified to allow
NFS4ERR_DELAY to be returned by RELEASE_LOCKOWNER. However, the
scope of this addition is limited and is not to be considered as
making this error return generally acceptable.
It needs to be made clear that servers may not return this error to
clients not prepared to support filesystem migration. Such clients
may be following the error specifications in [RFC3530] and
[cur-rfc3530-bis] and so might not expect NFS4ERR_DELAY to be
returned on RELEASE_LOCKOWNER.
The following constraint applies to this additional error return, as
if it were a note appearing together with the newly allowed error
code:
In order to make server state fixed for a filesystem being
migrated, a server MAY return NFS4ERR_DELAY in response to a
RELEASE_LOCKOWNER that will affect locking state being propagated
to a destination server. The source server MUST NOT do so unless
it is likely that it will later return NFS4ERR_MOVED for the
filesystem in question.
In the context of lockowner release, the set of filesystems such
that server state being made fixed can result in NFS4ERR_DELAY
must include the filesystem on which the operation associated with
the current lockowner seqid was performed.
In addition, this set may include other filesystems on which an
operation associated with an earlier seqid for the current
lockowner seqid was performed, since servers will have to deal
with the issue of an owner being used in succession for multiple
filesystems.
Thus, a client that is prepared to receive NFS4ERR_MOVED after
creating state associated with a given filesystem, it also needs
to be prepared to receive NFS4ERR_DELAY in response to
RELEASE_LOCKOWNER, if it has used that owner in connection with a
file on that filesystem.
7.4. Operation 35: SETCLIENTID - Negotiate Client ID
7.4.1. SYNOPSIS
client, callback, callback_ident -> clientid, setclientid_confirm client, callback, callback_ident -> clientid, setclientid_confirm
7.3.2. ARGUMENT 7.4.2. ARGUMENT
struct SETCLIENTID4args { struct SETCLIENTID4args {
nfs_client_id4 client; nfs_client_id4 client;
cb_client4 callback; cb_client4 callback;
uint32_t callback_ident; uint32_t callback_ident;
}; };
7.3.3. RESULT 7.4.3. RESULT
struct SETCLIENTID4resok { struct SETCLIENTID4resok {
clientid4 clientid; clientid4 clientid;
verifier4 setclientid_confirm; verifier4 setclientid_confirm;
}; };
union SETCLIENTID4res switch (nfsstat4 status) { union SETCLIENTID4res switch (nfsstat4 status) {
case NFS4_OK: case NFS4_OK:
SETCLIENTID4resok resok4; SETCLIENTID4resok resok4;
case NFS4ERR_CLID_INUSE: case NFS4ERR_CLID_INUSE:
clientaddr4 client_using; clientaddr4 client_using;
default: default:
void; void;
}; };
7.3.4. DESCRIPTION 7.4.4. DESCRIPTION
The client uses the SETCLIENTID operation to notify the server of its The client uses the SETCLIENTID operation to notify the server of its
intention to use a particular client identifier, callback, and intention to use a particular client identifier, callback, and
callback_ident for subsequent requests that entail creating lock, callback_ident for subsequent requests that entail creating lock,
share reservation, and delegation state on the server. Upon share reservation, and delegation state on the server. Upon
successful completion the server will return a shorthand client ID successful completion the server will return a shorthand client ID
which, if confirmed via a separate step, will be used in subsequent which, if confirmed via a separate step, will be used in subsequent
file locking and file open requests. Confirmation of the client ID file locking and file open requests. Confirmation of the client ID
must be done via the SETCLIENTID_CONFIRM operation to return the must be done via the SETCLIENTID_CONFIRM operation to return the
client ID and setclientid_confirm values, as verifiers, to the client ID and setclientid_confirm values, as verifiers, to the
server. The reason why two verifiers are necessary is that it is server. The reason why two verifiers are necessary is that it is
skipping to change at page 39, line 29 skipping to change at page 41, line 25
The callback information provided in this operation will be used if The callback information provided in this operation will be used if
the client is provided an open delegation at a future point. the client is provided an open delegation at a future point.
Therefore, the client must correctly reflect the program and port Therefore, the client must correctly reflect the program and port
numbers for the callback program at the time SETCLIENTID is used. numbers for the callback program at the time SETCLIENTID is used.
The callback_ident value is used by the server on the callback. The The callback_ident value is used by the server on the callback. The
client can leverage the callback_ident to eliminate the need for more client can leverage the callback_ident to eliminate the need for more
than one callback RPC program number, while still being able to than one callback RPC program number, while still being able to
determine which server is initiating the callback. determine which server is initiating the callback.
7.3.5. IMPLEMENTATION 7.4.5. IMPLEMENTATION
To understand how to implement SETCLIENTID, make the following To understand how to implement SETCLIENTID, make the following
notations. Let: notations. Let:
x be the value of the client.id subfield of the SETCLIENTID4args x be the value of the client.id subfield of the SETCLIENTID4args
structure. structure.
v be the value of the client.verifier subfield of the v be the value of the client.verifier subfield of the
SETCLIENTID4args structure. SETCLIENTID4args structure.
skipping to change at page 40, line 10 skipping to change at page 41, line 50
callback_ident fields of the SETCLIENTID4args structure. callback_ident fields of the SETCLIENTID4args structure.
s be the setclientid_confirm value returned in the SETCLIENTID4resok s be the setclientid_confirm value returned in the SETCLIENTID4resok
structure. structure.
{ v, x, c, k, s } be a quintuple for a client record. A client { v, x, c, k, s } be a quintuple for a client record. A client
record is confirmed if there has been a SETCLIENTID_CONFIRM record is confirmed if there has been a SETCLIENTID_CONFIRM
operation to confirm it. Otherwise it is unconfirmed. An operation to confirm it. Otherwise it is unconfirmed. An
unconfirmed record is established by a SETCLIENTID call. unconfirmed record is established by a SETCLIENTID call.
7.3.5.1. IMPLEMENTATION (preparatory phase) 7.4.5.1. IMPLEMENTATION (preparatory phase)
Since SETCLIENTID is a non-idempotent operation, let us assume that Since SETCLIENTID is a non-idempotent operation, let us assume that
the server is implementing the duplicate request cache (DRC). the server is implementing the duplicate request cache (DRC).
When the server gets a SETCLIENTID { v, x, k } request, it first does When the server gets a SETCLIENTID { v, x, k } request, it first does
a number of preliminary checks as listed below before proceeding to a number of preliminary checks as listed below before proceeding to
the main part of SETCLIENTID processing. the main part of SETCLIENTID processing.
o It first looks up the request in the DRC. If there is a hit, it o It first looks up the request in the DRC. If there is a hit, it
returns the result cached in the DRC. The server does NOT remove returns the result cached in the DRC. The server does NOT remove
client state (locks, shares, delegations) nor does it modify any client state (locks, shares, delegations) nor does it modify any
skipping to change at page 40, line 46 skipping to change at page 42, line 38
o Otherwise, the server is being asked to do a SETCLIENTID for a o Otherwise, the server is being asked to do a SETCLIENTID for a
client by a non-matching principal while there is active state and client by a non-matching principal while there is active state and
the server rejects the SETCLIENTID request returning an the server rejects the SETCLIENTID request returning an
NFS4ERR_CLID_INUSE error, since use of a single client with NFS4ERR_CLID_INUSE error, since use of a single client with
multiple principals is not allowed. Note that even though the multiple principals is not allowed. Note that even though the
previously used clientaddr is returned with this error, the use of previously used clientaddr is returned with this error, the use of
the same id string with multiple clientaddr's is not prohibited, the same id string with multiple clientaddr's is not prohibited,
while its use with multiple principals is prohibited. while its use with multiple principals is prohibited.
7.3.5.2. IMPLEMENTATION (main phase) 7.4.5.2. IMPLEMENTATION (main phase)
If the SETCLIENTID has not been dealt with by DRC processing, and has If the SETCLIENTID has not been dealt with by DRC processing, and has
not been rejected with an NFS4ERR_CLID_INUSE error, then the main not been rejected with an NFS4ERR_CLID_INUSE error, then the main
part of SETCLIENTID processing proceeds, as described below. part of SETCLIENTID processing proceeds, as described below.
o The server checks if it has recorded a confirmed record for { v, o The server checks if it has recorded a confirmed record for { v,
x, c, l, s }, where l may or may not equal k. If so, and since the x, c, l, s }, where l may or may not equal k. If so, and since the
id verifier v of the request matches that which is confirmed and id verifier v of the request matches that which is confirmed and
recorded, the server treats this as a probable callback recorded, the server treats this as a probable callback
information update and records an unconfirmed { v, x, c, k, t } information update and records an unconfirmed { v, x, c, k, t }
skipping to change at page 42, line 25 skipping to change at page 44, line 22
The server returns { d, t }. The server returns { d, t }.
The server awaits confirmation of { d, k } via SETCLIENTID_CONFIRM The server awaits confirmation of { d, k } via SETCLIENTID_CONFIRM
{ d, t }. The server does NOT remove client (lock/share/ { d, t }. The server does NOT remove client (lock/share/
delegation) state for x. delegation) state for x.
The server generates the clientid and setclientid_confirm values and The server generates the clientid and setclientid_confirm values and
must take care to ensure that these values are extremely unlikely to must take care to ensure that these values are extremely unlikely to
ever be regenerated. ever be regenerated.
7.4. Security Considerations revision 7.5. Security Considerations revision
The last paragraph of the "Security Considerations" section should be The last paragraph of the "Security Considerations" section should be
revised to read as follows: revised to read as follows:
Because the operations SETCLIENTID/SETCLIENTID_CONFIRM are Because the operations SETCLIENTID/SETCLIENTID_CONFIRM are
responsible for the release of client state, it is imperative that responsible for the release of client state, it is imperative that
the principal used for these operations is checked against and the principal used for these operations is checked against and
match the previous use of these operations. In addition, use of match the previous use of these operations. In addition, use of
integrity protection is desirable on the SETCLIENTID operation, to integrity protection is desirable on the SETCLIENTID operation, to
prevent an attack whereby a change in the boot verifier forces an prevent an attack whereby a change in the boot verifier forces an
undesired loss of client state. See the section "Client Identity undesired loss of client state. See the section "Client Identity
Definition" for further discussion. Definition" for further discussion.
8. Security Considerations 8. Security Considerations
Is modified as specified in Section 7.4. Is modified as specified in Section 7.5.
9. IANA Considerations 9. IANA Considerations
This document does not require actions by IANA. This document does not require actions by IANA.
10. Acknowledgements 10. Acknowledgements
The editor and authors of this document gratefully acknowledge the The editor and authors of this document gratefully acknowledge the
contributions of Trond Myklebust of NetApp and Robert Thurlow of contributions of Trond Myklebust of NetApp and Robert Thurlow of
Oracle. We also thank Tom Haynes of NetApp and Spencer Shepler of Oracle. We also thank Tom Haynes of NetApp and Spencer Shepler of
 End of changes. 42 change blocks. 
108 lines changed or deleted 214 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/