draft-ietf-nfsv4-migration-issues-10.txt   draft-ietf-nfsv4-migration-issues-11.txt 
NFSv4 D. Noveck, Ed. NFSv4 D. Noveck, Ed.
Internet-Draft HPE Internet-Draft
Intended status: Informational P. Shivam Intended status: Informational P. Shivam
Expires: February 18, 2017 C. Lever Expires: August 12, 2017 C. Lever
B. Baker B. Baker
ORACLE ORACLE
August 17, 2016 February 8, 2017
NFSv4 migration: Implementation Experience and Specification Issues NFSv4 migration: Implementation Experience and Specification Issues
draft-ietf-nfsv4-migration-issues-10 draft-ietf-nfsv4-migration-issues-11
Abstract Abstract
The migration feature of NFSv4 provides for moving responsibility for The migration feature of NFSv4 provides for moving responsibility for
a single filesystem from one server to another, without disruption to a single filesystem from one server to another, without disruption to
clients. Recent implementation experience has shown problems in the clients. Implementation experience has shown problems in
existing specification for this feature. This document discusses specification for this feature in RFC7530. This document explains
options to cure issues which have arisen. It also explains the the choices made to address these issues by updating the NFSv4.0
choices made in updating the NFSv4.0 specification and those to be specification in RFC7931 and those to be made with regard to the
made with regard to the NFSv4.1 specification, in order to properly NFSv4.1 specification, in order to properly address migration.
address migration.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 18, 2017. This Internet-Draft will expire on August 12, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. NFSv4.0 Implementation Experience . . . . . . . . . . . . . . 4 3. NFSv4.0 Implementation Experience . . . . . . . . . . . . . . 3
3.1. Implementation Issues . . . . . . . . . . . . . . . . . . 4 3.1. Implementation Issues . . . . . . . . . . . . . . . . . . 3
3.1.1. Failure to Free Migrated State on Client Reboot . . . 4 3.1.1. Failure to Free Migrated State on Client Reboot . . . 4
3.1.2. Server Reboots Resulting in a Confused Lease 3.1.2. Server Reboots Resulting in a Confused Lease
Situation . . . . . . . . . . . . . . . . . . . . . . 5 Situation . . . . . . . . . . . . . . . . . . . . . . 4
3.1.3. Client Complexity Issues . . . . . . . . . . . . . . 6 3.1.3. Client Complexity Issues . . . . . . . . . . . . . . 5
3.2. Sources of Protocol Difficulties . . . . . . . . . . . . 7 3.2. Sources of Protocol Difficulties . . . . . . . . . . . . 7
3.2.1. Issues with nfs_client_id4 Generation and Use . . . . 7 3.2.1. Issues with nfs_client_id4 Generation and Use . . . . 7
3.2.2. Issues with Lease Proliferation . . . . . . . . . . . 9 3.2.2. Issues with Lease Proliferation . . . . . . . . . . . 9
4. Issues Requiring Resolution in NFSv4.0 . . . . . . . . . . . 10 4. Resolution of NFSv4.0 Protocol Difficulties . . . . . . . . . 9
4.1. Changes to nfs_client_id4 Client-string . . . . . . . . . 10 4.1. Changes Regarding nfs_client_id4 Client-string . . . . . 9
4.2. Changes to Handle Differing nfs_client_id4 String Values 11 4.2. Changes Regarding Merged (vs. Synchronized) Leases . . . 10
4.3. Potential Changes to Add a New Operation . . . . . . . . 11 4.3. Other Changes to Migration-state Sections . . . . . . . . 11
4.4. Other Issues Within Migration-state Sections . . . . . . 12 4.3.1. Changes Regarding Client ID Migration . . . . . . . . 12
4.5. Issues Within Other Sections . . . . . . . . . . . . . . 12 4.3.2. Changes Regarding Callback Re-establishment . . . . . 12
5. Resolution of NFSv4.0 Protocol Difficulties . . . . . . . . . 13 4.3.3. NFS4ERR_LEASE_MOVED Rework . . . . . . . . . . . . . 13
5.1. Changes Regarding nfs_client_id4 Client-string . . . . . 13 4.4. Changes to Other Sections . . . . . . . . . . . . . . . . 13
5.2. Changes Regarding Merged (vs. Synchronized) Leases . . . 14 4.4.1. Need for Additional Changes . . . . . . . . . . . . . 13
5.3. Other Changes to Migration-state Sections . . . . . . . . 15 4.4.2. Callback Update . . . . . . . . . . . . . . . . . . . 14
5.3.1. Changes Regarding Client ID Migration . . . . . . . . 15 4.4.3. clientid4 Handling . . . . . . . . . . . . . . . . . 14
5.3.2. Changes Regarding Callback Re-establishment . . . . . 16 4.4.4. Handling of NFS4ERR_CLID_INUSE . . . . . . . . . . . 16
5.3.3. NFS4ERR_LEASE_MOVED Rework . . . . . . . . . . . . . 16 5. Issues for NFSv4.1 . . . . . . . . . . . . . . . . . . . . . 17
5.4. Changes to Other Sections . . . . . . . . . . . . . . . . 17 5.1. Addressing state merger in NFSv4.1 . . . . . . . . . . . 17
5.4.1. Callback Update . . . . . . . . . . . . . . . . . . . 17 5.2. Addressing pNFS relationship with migration . . . . . . . 18
5.4.2. clientid4 Handling . . . . . . . . . . . . . . . . . 17 5.3. Addressing server owner changes in NFSv4.1 . . . . . . . 18
5.4.3. Handling of NFS4ERR_CLID_INUSE . . . . . . . . . . . 19 6. Security Considerations . . . . . . . . . . . . . . . . . . . 19
6. Issues for NFSv4.1 . . . . . . . . . . . . . . . . . . . . . 20 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
6.1. Addressing state merger in NFSv4.1 . . . . . . . . . . . 20 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20
6.2. Addressing pNFS relationship with migration . . . . . . . 21 9. Normative References . . . . . . . . . . . . . . . . . . . . 20
6.3. Addressing server owner changes in NFSv4.1 . . . . . . . 21 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20
7. Security Considerations . . . . . . . . . . . . . . . . . . . 22
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23
10.1. Normative References . . . . . . . . . . . . . . . . . . 23
10.2. Informative References . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction 1. Introduction
This document is in the informational category, and while the facts This document is in the informational category, and while the facts
it reports may have normative implications, any such normative it reports may have normative implications, any such normative
significance reflects the readers' preferences. For example, we may significance reflects the readers' preferences. For example, we may
report that the reboot of a client with migrated state results in report that the reboot of a client with migrated state results in
state not being promptly cleared and that this will prevent granting state not being promptly cleared and that this will prevent granting
of conflicting lock requests at least for the lease time, which is a of conflicting lock requests at least for the lease time, which is a
fact. While it is to be expected that client and server implementers fact. While it is to be expected that client and server implementers
skipping to change at page 5, line 43 skipping to change at page 5, line 35
callback update. callback update.
The SETCLIENTID for callback update only includes the nfs_client_id4, The SETCLIENTID for callback update only includes the nfs_client_id4,
assuming there can only be one such with a given nfs_client_id4 assuming there can only be one such with a given nfs_client_id4
value. If there were multiple, confirmed client records with value. If there were multiple, confirmed client records with
identical nfs_client_id4 id string values, there would be no way to identical nfs_client_id4 id string values, there would be no way to
map the callback update request to the correct client record. Apart map the callback update request to the correct client record. Apart
from the migration handling specified in [RFC7530], such a situation from the migration handling specified in [RFC7530], such a situation
cannot arise. cannot arise.
One possible accommodation for this particular issue that has been
used is to add a RENEW operation along with SETCLIENTID (on a
callback update) to disambiguate the client.
When the client updates the callback info to the destination, the
client would, by convention, send a compound like this:
{ RENEW clientid4, SETCLIENTID nfs_client_id4,verf,cb }
The presence of the clientid4 in the compound would allow the server
to differentiate among the various leases that it knows of, all with
the same nfs_client_id4 value.
While this would be a reasonable patch for an isolated protocol
weakness, interoperable clients and servers would require that the
protocol truly be updated to allow such a situation, specifically
that of multiple clientid4's with the same nfs_client_id4 value. The
protocol is currently designed and implemented assuming this cannot
happen. We need to either prevent the situation from happening, or
fully adapt to the possibilities which can arise. See Section 4 for
a discussion of such issues.
3.1.3. Client Complexity Issues 3.1.3. Client Complexity Issues
Consider the following situation: Consider the following situation:
o There are a set of clients C1 through Cn accessing servers S1 o There are a set of clients C1 through Cn accessing servers S1
through Sm. Each server manages some significant number of through Sm. Each server manages some significant number of
filesystems with the filesystem count L being significantly filesystems with the filesystem count L being significantly
greater than m. greater than m.
o Each client Cx will access a subset of the servers and so will o Each client Cx will access a subset of the servers and so will
skipping to change at page 10, line 21 skipping to change at page 9, line 39
for clients supporting the migration feature. for clients supporting the migration feature.
Because of the discussion of client-string construction in [RFC7530], Because of the discussion of client-string construction in [RFC7530],
most existing clients implement the non-uniform client-string most existing clients implement the non-uniform client-string
approach. As a result, existing servers may not have been tested approach. As a result, existing servers may not have been tested
with clients implementing uniform client-strings. As a consequence, with clients implementing uniform client-strings. As a consequence,
care must be taken to preserve interoperability between UCS-capable care must be taken to preserve interoperability between UCS-capable
clients and servers that don't tolerate uniform client strings for clients and servers that don't tolerate uniform client strings for
one reason or another. one reason or another.
4. Issues Requiring Resolution in NFSv4.0 4. Resolution of NFSv4.0 Protocol Difficulties
4.1. Changes to nfs_client_id4 Client-string
The fact that the reason given in client-string-BP3 is not valid
makes the existing "should" insupportable. We can't either
o Keep a reason we know is invalid.
o Keep saying "should" without giving a reason.
What are often presented as reasons that motivate use of the non-
uniform approach always turn out to be cases in which, if the uniform
approach were used, the server will treat a client which accesses
that server via two different IP addresses as part of a single
client, as it in fact is. This may be disconcerting to a client
unaware that the two IP addresses connect to the same server. This
is not a reason to use the non-uniform approach but is better thought
of as an illustration of the fact that those using the uniform
approach need to be aware of the possibility of server trunking and
its potential effect on server behavior.
Since it is possible to reliably infer the existence of trunking of
server IP addresses from observed server behavior, use of the uniform
approach is more desirable, although compatibility issues need to be
dealt with.
An alternative to having the client infer the existence of trunking
of IP server addresses, is to make this information available to the
client directly. See Section 4.3 for details.
It is always possible that a valid new reason will be found, but so
far none has been proposed. Given the history, the burden of proof
was on those asserting the validity of a proposed new reason.
So we will assume that the "should" needs to go. The question was
what to replace it with.
o We can't say "MUST NOT", despite the problems this raises for
migration since this is pretty late in the day for such a change.
Many currently operating clients obey the existing "should".
Similar considerations would apply for "SHOULD NOT" or "should
not".
o Dropping client-string-BP3 entirely is a possibility but, given
the context and history, it would just be a confusing version of
"SHOULD NOT".
o Using "MAY" would clearly specify that both ways of doing this are
valid choices for clients and that servers will have to deal with
clients that make either choice.
o This might be modified by a "SHOULD" (or even a "MUST") for
particular groups of clients.
o There has to be some text explaining why a client might make
either choice but, except for the particular cases referred to
above, it will have to made sure that it is truly descriptive, and
not slanted in either direction.
4.2. Changes to Handle Differing nfs_client_id4 String Values
Given the difficulties caused by having different nfs_client_id4
client-string values for the same client, we had two choices:
o Deprecating the existing treatment and basically saying the client
is on its own doing migration, if it follows it.
o Introducing a way of having the client provide client identity
information to the server, if it can be done compatibly while
staying within the bounds of v4.0.
4.3. Potential Changes to Add a New Operation
It might be possible to return server-identity information to the
client, just as is done in NFSv4.1 by the response to the EXCHANGE_ID
operation. This could be done by a SETCLIENTID_PLUS optional
operation, which acts like SETCLIENTID, except that it returns server
identity information. Such information could be used by clients,
making it possible to for them to be aware of server trunking
relationships, rather than having to infer them from server behavior.
It has been generally thought that protocol extensions such as this
are not appropriate in bis documents and other documents updating
NFSv4 protocol definition RFC's. However, [NFSv4-vers] discusses
means by which protocol extensions, similar to those allowed between
minor versions, could be used to correct protocol mistakes.
A decision to adopt this approach would require waiting for
[NFSv4-vers] to become a Proposed Standard. In view of the time
necessary for that to happen, this approach was not available in an
RFC updating [RFC7530], such as [RFC7931]. Still, it is worth
keeping in mind, if implementers have difficulties inferring trunking
relationships using the techniques discussed there.
4.4. Other Issues Within Migration-state Sections
There are a number of issues where the existing text is unclear and/
or wrong and needs to be fixed in some way.
o Lack of clarity in the discussion of moving clientids (as well as
stateids) as part of moving state for migration.
o The discussion of synchronized leases is wrong in that there is no
way to determine (in the current spec) when leases are for the
same client and also wrong in suggesting a benefit from leases
synchronized at the point of transfer. What is needed is merger
of leases, which is necessary to keep client complexity
requirements from getting out of hand.
o Lack of clarity in the discussion of LEASE_MOVED handling,
including failure to fully address situations in which transparent
state migration did not occur.
4.5. Issues Within Other Sections
There are a number of cases in which certain sections, not
specifically related to migration, require additional clarification.
This is generally because text that is clear in a context in which
leases and clientids are created in one place and live there forever
may need further refinement in the more dynamic environment that
arises as part of migration.
Some examples:
o Some people are under the impression that updating callback
endpoint information for an existing client, as used during
migration, may cause the destination server to free existing
state. There need to be additions to clarify the situation.
o The handling of the sets of clientid4's maintained by each server
needs to be clarified. In particular, the issue of how the client
adapts to the presumably independent and uncoordinated clientid4
sets needs to be clearly addressed
o Statements regarding handling of invalid clientid4's need to be
clarified and/or refined in light of the possibilities that arise
due to lease motion and merger.
o Confusion and lack of clarity about NFS4ERR_CLID_INUSE.
5. Resolution of NFSv4.0 Protocol Difficulties
This section lists the changes that were necessary to resolve the This section lists the changes that were necessary to resolve the
difficulties mentioned above. Such changes, along with other difficulties mentioned above. Such changes, along with other
clarifications found to be desirable during drafting and review are clarifications found to be desirable during drafting and review are
contained in [RFC7931]. contained in [RFC7931].
5.1. Changes Regarding nfs_client_id4 Client-string 4.1. Changes Regarding nfs_client_id4 Client-string
It was decided to replace client-string-BP3 with the following text: It was decided to replace client-string-BP3 with the following text:
The string MAY be different for each server network address that The string MAY be different for each server network address that
the client accesses, rather than common to all server network the client accesses, rather than common to all server network
addresses. addresses.
In addition, given the importance of the issue of client identity and In addition, given the importance of the issue of client identity and
the fact that both client string-approaches are to be considered the fact that both client string-approaches are to be considered
valid, a greatly expanded treatment of client identity was desirable. valid, a greatly expanded treatment of client identity was desirable.
skipping to change at page 14, line 8 skipping to change at page 10, line 28
o Describing the compatibility issues that might cause servers to be o Describing the compatibility issues that might cause servers to be
incompatible with the uniform approach and give guidance about incompatible with the uniform approach and give guidance about
dealing with these. dealing with these.
o Describing how a client using the uniform approach might use o Describing how a client using the uniform approach might use
server behavior to determine server address trunking patterns. server behavior to determine server address trunking patterns.
o Presenting a clearer and more complete set of recommendations to o Presenting a clearer and more complete set of recommendations to
guide client string construction. guide client string construction.
5.2. Changes Regarding Merged (vs. Synchronized) Leases 4.2. Changes Regarding Merged (vs. Synchronized) Leases
In [RFC7530], the section entitled "Migration and State" says: In [RFC7530], the section entitled "Migration and State" says:
As part of the transfer of information between servers, leases As part of the transfer of information between servers, leases
would be transferred as well. The leases being transferred to the would be transferred as well. The leases being transferred to the
new server will typically have a different expiration time from new server will typically have a different expiration time from
those for the same client, previously on the old server. To those for the same client, previously on the old server. To
maintain the property that all leases on a given server for a maintain the property that all leases on a given server for a
given client expire at the same time, the server should advance given client expire at the same time, the server should advance
the expiration time to the later of the leases being transferred the expiration time to the later of the leases being transferred
skipping to change at page 15, line 26 skipping to change at page 11, line 46
If servers obey the SHOULD and clients choose to adopt the uniform id If servers obey the SHOULD and clients choose to adopt the uniform id
approach, having more than a single lease for a given client-server approach, having more than a single lease for a given client-server
pair will be a transient situation, cleaned up as part of adapting to pair will be a transient situation, cleaned up as part of adapting to
use of migrated state. use of migrated state.
Since clients and servers will be a mixture of old and new and Since clients and servers will be a mixture of old and new and
because nothing is a MUST we have to ensure that no combination will because nothing is a MUST we have to ensure that no combination will
show worse behavior than is exhibited by current (i.e. old) clients show worse behavior than is exhibited by current (i.e. old) clients
and servers. and servers.
5.3. Other Changes to Migration-state Sections 4.3. Other Changes to Migration-state Sections
4.3.1. Changes Regarding Client ID Migration
5.3.1. Changes Regarding Client ID Migration
In [RFC7530], the section entitled "Migration and State" says: In [RFC7530], the section entitled "Migration and State" says:
In the case of migration, the servers involved in the migration of In the case of migration, the servers involved in the migration of
a filesystem SHOULD transfer all server state from the original to a filesystem SHOULD transfer all server state from the original to
the new server. This must be done in a way that is transparent to the new server. This must be done in a way that is transparent to
the client. This state transfer will ease the client's transition the client. This state transfer will ease the client's transition
when a filesystem migration occurs. If the servers are successful when a filesystem migration occurs. If the servers are successful
in transferring all state, the client will continue to use in transferring all state, the client will continue to use
stateids assigned by the original server. Therefore the new stateids assigned by the original server. Therefore the new
skipping to change at page 16, line 24 skipping to change at page 12, line 46
We have decided that it is best to address this issue as follows: We have decided that it is best to address this issue as follows:
o Make it clear that both clientid4 and nfs_client_id4 (including o Make it clear that both clientid4 and nfs_client_id4 (including
both id string and boot verifier) are to be transferred. both id string and boot verifier) are to be transferred.
o Indicate that the initial transfer will result in the same o Indicate that the initial transfer will result in the same
clientid4 after transfer but this is not guaranteed since there clientid4 after transfer but this is not guaranteed since there
may conflict with an existing clientid4 on the destination server may conflict with an existing clientid4 on the destination server
and because lease merger can result in a change of the clientid4. and because lease merger can result in a change of the clientid4.
5.3.2. Changes Regarding Callback Re-establishment 4.3.2. Changes Regarding Callback Re-establishment
In [RFC7530], the section entitled "Migration and State" says: In [RFC7530], the section entitled "Migration and State" says:
A client SHOULD re-establish new callback information with the new A client SHOULD re-establish new callback information with the new
server as soon as possible, according to sequences described in server as soon as possible, according to sequences described in
sections "Operation 35: SETCLIENTID - Negotiate Client ID" and sections "Operation 35: SETCLIENTID - Negotiate Client ID" and
"Operation 36: SETCLIENTID_CONFIRM - Confirm Client ID". This "Operation 36: SETCLIENTID_CONFIRM - Confirm Client ID". This
ensures that server operations are not blocked by the inability to ensures that server operations are not blocked by the inability to
recall delegations. recall delegations.
The above will need to be fixed to reflect the possibility of merging The above will need to be fixed to reflect the possibility of merging
of leases, of leases,
5.3.3. NFS4ERR_LEASE_MOVED Rework 4.3.3. NFS4ERR_LEASE_MOVED Rework
In [RFC7530], the section entitled "Notification of Migrated Lease" In [RFC7530], the section entitled "Notification of Migrated Lease"
says: says:
Upon receiving the NFS4ERR_LEASE_MOVED error, a client that Upon receiving the NFS4ERR_LEASE_MOVED error, a client that
supports filesystem migration MUST probe all filesystems from that supports filesystem migration MUST probe all filesystems from that
server on which it holds open state. Once the client has server on which it holds open state. Once the client has
successfully probed all those filesystems which are migrated, the successfully probed all those filesystems which are migrated, the
server MUST resume normal handling of stateful requests from that server MUST resume normal handling of stateful requests from that
client. client.
skipping to change at page 17, line 13 skipping to change at page 13, line 36
must be. This has led to some worry about the scalability of the must be. This has led to some worry about the scalability of the
probing process, and although the time required does scale linearly probing process, and although the time required does scale linearly
with the number of filesystems that the client may have state for with the number of filesystems that the client may have state for
with respect to a given server, the actual process can be done with respect to a given server, the actual process can be done
efficiently. efficiently.
To address these issues, the text above had to be rewritten to be To address these issues, the text above had to be rewritten to be
more clear and to give suggestions about how to do the required more clear and to give suggestions about how to do the required
scanning efficiently. scanning efficiently.
5.4. Changes to Other Sections 4.4. Changes to Other Sections
5.4.1. Callback Update 4.4.1. Need for Additional Changes
There are a number of cases in which certain sections, not
specifically related to migration, require additional clarification.
This is generally because text that is clear in a context in which
leases and clientids are created in one place and live there forever
may need further refinement in the more dynamic environment that
arises as part of migration.
Some examples:
o Some people are under the impression that updating callback
endpoint information for an existing client, as used during
migration, may cause the destination server to free existing
state. There need to be additions to clarify the situation.
o The handling of the sets of clientid4's maintained by each server
needs to be clarified. In particular, the issue of how the client
adapts to the presumably independent and uncoordinated clientid4
sets needs to be clearly addressed
o Statements regarding handling of invalid clientid4's need to be
clarified and/or refined in light of the possibilities that arise
due to lease motion and merger.
o Confusion and lack of clarity about NFS4ERR_CLID_INUSE.
4.4.2. Callback Update
Some changes are necessary to reduce confusion about the process of Some changes are necessary to reduce confusion about the process of
callback information update and in particular to make it clear that callback information update and in particular to make it clear that
no state is freed as a result: no state is freed as a result:
o Make it clear that after migration there are confirmed entries for o Make it clear that after migration there are confirmed entries for
transferred clientid4/nfs_client_id4 pairs. transferred clientid4/nfs_client_id4 pairs.
o Be explicit in the sections headed "otherwise," in the o Be explicit in the sections headed "otherwise," in the
descriptions of SETCLIENTID and SETCLIENTID_CONFIRM, that these descriptions of SETCLIENTID and SETCLIENTID_CONFIRM, that these
don't apply in the cases we are concerned about. don't apply in the cases we are concerned about.
5.4.2. clientid4 Handling 4.4.3. clientid4 Handling
To address both of the clientid4-related issues mentioned in To address both of the clientid4-related issues mentioned in
Section 4.5, it was necessary to replace the last three paragraphs of Section 4.4.1, it was necessary to replace the last three paragraphs
the section entitled "Client ID" with the following: of the section entitled "Client ID" with the following:
Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has
successfully completed, the client uses the shorthand client successfully completed, the client uses the shorthand client
identifier, of type clientid4, instead of the longer and less identifier, of type clientid4, instead of the longer and less
compact nfs_client_id4 structure. This shorthand client compact nfs_client_id4 structure. This shorthand client
identifier (a client ID) is assigned by the server and should be identifier (a client ID) is assigned by the server and should be
chosen so that it will not conflict with a client ID previously chosen so that it will not conflict with a client ID previously
assigned by same server. This applies across server restarts or assigned by same server. This applies across server restarts or
reboots. reboots.
skipping to change at page 19, line 11 skipping to change at page 16, line 13
receives a NFS4ERR_STALE_STATEID error using a stateid derived receives a NFS4ERR_STALE_STATEID error using a stateid derived
from its current clientid4, since this indicates a situation, such from its current clientid4, since this indicates a situation, such
as server reboot which has invalidated the existing clientid4 and as server reboot which has invalidated the existing clientid4 and
associated stateids (see the section entitled "lock-owner" for associated stateids (see the section entitled "lock-owner" for
details). details).
See the detailed descriptions of SETCLIENTID and See the detailed descriptions of SETCLIENTID and
SETCLIENTID_CONFIRM for a complete specification of the SETCLIENTID_CONFIRM for a complete specification of the
operations. operations.
5.4.3. Handling of NFS4ERR_CLID_INUSE 4.4.4. Handling of NFS4ERR_CLID_INUSE
It appears to be the intention that only a single principal be used It appears to be the intention that only a single principal be used
for client establishment between any client-server pair. However: for client establishment between any client-server pair. However:
o There is no explicit statement to this effect. o There is no explicit statement to this effect.
o The error that indicates a principal conflict has a name which o The error that indicates a principal conflict has a name which
does not clarify this issue: NFS4ERR_CLID_INUSE. does not clarify this issue: NFS4ERR_CLID_INUSE.
o The definition of the error is also not very helpful: "The o The definition of the error is also not very helpful: "The
skipping to change at page 20, line 5 skipping to change at page 17, line 5
principal and that client instance currently holds an active principal and that client instance currently holds an active
lease. A server MAY return this error if the same principal is lease. A server MAY return this error if the same principal is
used but a change in authentication flavor gives good reason to used but a change in authentication flavor gives good reason to
reject the new SETCLIENTID operation as not bona fide. reject the new SETCLIENTID operation as not bona fide.
o In the description of SETCLIENTID, the phrase "then the server o In the description of SETCLIENTID, the phrase "then the server
returns a NFS4ERR_CLID_INUSE error" should be expanded to read returns a NFS4ERR_CLID_INUSE error" should be expanded to read
"then the server returns a NFS4ERR_CLID_INUSE error, since use of "then the server returns a NFS4ERR_CLID_INUSE error, since use of
a single client with multiple principals is not allowed." a single client with multiple principals is not allowed."
6. Issues for NFSv4.1 5. Issues for NFSv4.1
Because NFSv4.1 embraces the uniform client-string approach, as Because NFSv4.1 embraces the uniform client-string approach, as
advised by section 2.4 of [RFC5661], addressing migration issues is advised by section 2.4 of [RFC5661], addressing migration issues is
simpler. simpler.
Nevertheless, there are some issues that will have to be addressed. Nevertheless, there are some issues that will have to be addressed.
Some examples: Some examples:
o The other necessary part of addressing migration issues, providing o The other necessary part of addressing migration issues, providing
for the server's merger of leases that relate to the same client, for the server's merger of leases that relate to the same client,
skipping to change at page 20, line 31 skipping to change at page 17, line 31
o There needs to be some clarification of how migration, and o There needs to be some clarification of how migration, and
particularly transparent state migration, should interact with particularly transparent state migration, should interact with
pNFS layouts. pNFS layouts.
o The current discussion (in [RFC5661]), of the possibility of o The current discussion (in [RFC5661]), of the possibility of
server_owner changes is incomplete and confusing. server_owner changes is incomplete and confusing.
Discussion of how to resolve these issues will appear in the sections Discussion of how to resolve these issues will appear in the sections
below. below.
6.1. Addressing state merger in NFSv4.1 5.1. Addressing state merger in NFSv4.1
The existing treatment of state transfer in [RFC5661], has similar The existing treatment of state transfer in [RFC5661], has similar
problems to that in [RFC7530] in that it assumes that the state for problems to that in [RFC7530] in that it assumes that the state for
multiple filesystems on different servers will not be merged to so multiple filesystems on different servers will not be merged to so
that it appears under a single common clientid. We've already seen that it appears under a single common clientid. We've already seen
the reasons that this is a problem, with regard to NFSv4.0. the reasons that this is a problem, with regard to NFSv4.0.
Although we don't have the problems stemming from the non-uniform Although we don't have the problems stemming from the non-uniform
client-string approach, there are a number of complexities in the client-string approach, there are a number of complexities in the
existing treatment of state management in the section entitled "Lock existing treatment of state management in the section entitled "Lock
skipping to change at page 21, line 12 skipping to change at page 18, line 12
o There is separate handling and discussion of the cases of matching o There is separate handling and discussion of the cases of matching
and non-matching server scopes. and non-matching server scopes.
o In the case of matching server scopes, the text calls for an o In the case of matching server scopes, the text calls for an
impossible degree of transparency. impossible degree of transparency.
o In the case of non-matching server scopes, the text does not o In the case of non-matching server scopes, the text does not
mention transparent state migration at all, resulting in a mention transparent state migration at all, resulting in a
functional regression from NFSV4.0 functional regression from NFSV4.0
6.2. Addressing pNFS relationship with migration 5.2. Addressing pNFS relationship with migration
This is made difficult because, within the PNFS framework, migration This is made difficult because, within the PNFS framework, migration
might mean any of several things: might mean any of several things:
o Transfer of the MDS, leaving DS's alone. o Transfer of the MDS, leaving DS's alone.
This would be minimally disruptive to those using layouts but This would be minimally disruptive to those using layouts but
would require the pNFS control protocol to support the DS being would require the pNFS control protocol to support the DS being
directed to a new MDS. directed to a new MDS.
skipping to change at page 21, line 37 skipping to change at page 18, line 37
o Transfer of the filesystem to a new filesystem with both MDS and o Transfer of the filesystem to a new filesystem with both MDS and
DS's moving. DS's moving.
In such a transfer, an entirely different set of DS's will be at In such a transfer, an entirely different set of DS's will be at
the target location. There may even be no pNFS support on the the target location. There may even be no pNFS support on the
destination filesystem at all. destination filesystem at all.
Migration needs to support both the first and last of these models. Migration needs to support both the first and last of these models.
6.3. Addressing server owner changes in NFSv4.1 5.3. Addressing server owner changes in NFSv4.1
Section 2.10.5 of [RFC5661] states the following. Section 2.10.5 of [RFC5661] states the following.
The client should be prepared for the possibility that The client should be prepared for the possibility that
eir_server_owner values may be different on subsequent EXCHANGE_ID eir_server_owner values may be different on subsequent EXCHANGE_ID
requests made to the same network address, as a result of various requests made to the same network address, as a result of various
sorts of reconfiguration events. When this happens and the sorts of reconfiguration events. When this happens and the
changes result in the invalidation of previously valid forms of changes result in the invalidation of previously valid forms of
trunking, the client should cease to use those forms, either by trunking, the client should cease to use those forms, either by
dropping connections or by adding sessions. For a discussion of dropping connections or by adding sessions. For a discussion of
skipping to change at page 22, line 40 skipping to change at page 19, line 40
eir_server_owner.so_major_id changes, the client can use eir_server_owner.so_major_id changes, the client can use
filehandles it has and attempt reclaims. It may find that filehandles it has and attempt reclaims. It may find that
these are now stale but if NFS4ERR_STALE is not received, he these are now stale but if NFS4ERR_STALE is not received, he
can proceed to reclaim his opens. can proceed to reclaim his opens.
* When eir_server_scope and eir_server_owner.so_major_id remain * When eir_server_scope and eir_server_owner.so_major_id remain
the same, the client has to use the now-current values of the same, the client has to use the now-current values of
eir_server-owner.so_minor_id in deciding on appropriate forms eir_server-owner.so_minor_id in deciding on appropriate forms
of trunking. of trunking.
7. Security Considerations 6. Security Considerations
With regard to NFSv4.0, the Security Considerations section of With regard to NFSv4.0, the Security Considerations section of
[RFC7530] encourages clients to protect the integrity of the SECINFO [RFC7530] encourages clients to protect the integrity of the SECINFO
operation, any GETATTR operation for the fs_locations attribute. A operation, any GETATTR operation for the fs_locations attribute. A
needed change is to include the operations SETCLIENTID/ needed change is to include the operations SETCLIENTID/
SETCLIENTID_CONFIRM as among those for which integrity protection is SETCLIENTID_CONFIRM as among those for which integrity protection is
recommended. A migration recovery event can use any or all of these recommended. A migration recovery event can use any or all of these
operations. operations.
With regard to NFSv4.1, the Security Considerations section of With regard to NFSv4.1, the Security Considerations section of
[RFC5661] takes proper care of migration-related issues. No change [RFC5661] takes proper care of migration-related issues. No change
is needed. is needed.
8. IANA Considerations 7. IANA Considerations
This document does not require actions by IANA. This document does not require actions by IANA.
9. Acknowledgements 8. Acknowledgements
The editor and authors of this document gratefully acknowledge the The editor and authors of this document gratefully acknowledge the
contributions of Trond Myklebust of NetApp and Robert Thurlow of contributions of Trond Myklebust of NetApp and Robert Thurlow of
Oracle. We also thank Tom Haynes of NetApp and Spencer Shepler of Oracle. We also thank Tom Haynes of NetApp and Spencer Shepler of
Microsoft for their guidance and suggestions. Microsoft for their guidance and suggestions.
Special thanks go to members of the Oracle Solaris NFS team, Special thanks go to members of the Oracle Solaris NFS team,
especially Rick Mesta and James Wahlig, for their work implementing especially Rick Mesta and James Wahlig, for their work implementing
an NFSv4.0 migration prototype and identifying many of the issues an NFSv4.0 migration prototype and identifying many of the issues
documented here. documented here.
10. References 9. Normative References
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
[RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed.,
"Network File System (NFS) Version 4 Minor Version 1 "Network File System (NFS) Version 4 Minor Version 1
Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010,
<http://www.rfc-editor.org/info/rfc5661>. <http://www.rfc-editor.org/info/rfc5661>.
[RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System
(NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530,
March 2015, <http://www.rfc-editor.org/info/rfc7530>. March 2015, <http://www.rfc-editor.org/info/rfc7530>.
[RFC7931] Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker, [RFC7931] Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker,
"NFSv4.0 Migration: Specification Update", RFC 7931, "NFSv4.0 Migration: Specification Update", RFC 7931,
DOI 10.17487/RFC7931, July 2016, DOI 10.17487/RFC7931, July 2016,
<http://www.rfc-editor.org/info/rfc7931>. <http://www.rfc-editor.org/info/rfc7931>.
10.2. Informative References
[NFSv4-vers]
Noveck, D., "NFSv4 Version Management", July 2016,
<http://www.ietf.org/id/
draft-ietf-nfsv4-versioning-05.txt>.
Work in progress.
Authors' Addresses Authors' Addresses
David Noveck (editor) David Noveck (editor)
Hewlett Packard Enterprise 26 Locust Avenue
165 Dascomb Road Lexington, MA 02421
Andover, MA 01810
US US
Phone: +1 978 474 2011 Phone: +1 781 572 8038
Email: davenoveck@gmail.com Email: davenoveck@gmail.com
Piyush Shivam Piyush Shivam
Oracle Corporation Oracle Corporation
5300 Riata Park Ct. 5300 Riata Park Ct.
Austin, TX 78727 Austin, TX 78727
US US
Phone: +1 512 401 1019 Phone: +1 512 401 1019
Email: piyush.shivam@oracle.com Email: piyush.shivam@oracle.com
 End of changes. 35 change blocks. 
245 lines changed or deleted 88 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/