draft-ietf-nfsv4-migration-issues-09.txt   draft-ietf-nfsv4-migration-issues-10.txt 
NFSv4 D. Noveck, Ed. NFSv4 D. Noveck, Ed.
Internet-Draft HPE Internet-Draft HPE
Intended status: Informational P. Shivam Intended status: Informational P. Shivam
Expires: August 22, 2016 C. Lever Expires: February 18, 2017 C. Lever
B. Baker B. Baker
ORACLE ORACLE
February 19, 2016 August 17, 2016
NFSv4 migration: Implementation experience and spec issues to resolve NFSv4 migration: Implementation Experience and Specification Issues
draft-ietf-nfsv4-migration-issues-09 draft-ietf-nfsv4-migration-issues-10
Abstract Abstract
The migration feature of NFSv4 provides for moving responsibility for The migration feature of NFSv4 provides for moving responsibility for
a single filesystem from one server to another, without disruption to a single filesystem from one server to another, without disruption to
clients. Recent implementation experience has shown problems in the clients. Recent implementation experience has shown problems in the
existing specification for this feature. This document discusses the existing specification for this feature. This document discusses
issues which have arisen and explores the options available for options to cure issues which have arisen. It also explains the
curing the issues. It also explains the choices made regarding choices made in updating the NFSv4.0 specification and those to be
updating the NFSv4.0 specification and those to be made with regard made with regard to the NFSv4.1 specification, in order to properly
to the NFSv4.1 specification, in order to properly address migration. address migration.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 22, 2016. This Internet-Draft will expire on February 18, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 16 skipping to change at page 2, line 16
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. NFSv4.0 Implementation Experience . . . . . . . . . . . . . . 4 3. NFSv4.0 Implementation Experience . . . . . . . . . . . . . . 4
3.1. Implementation issues . . . . . . . . . . . . . . . . . . 4 3.1. Implementation Issues . . . . . . . . . . . . . . . . . . 4
3.1.1. Failure to free migrated state on client reboot . . . 4 3.1.1. Failure to Free Migrated State on Client Reboot . . . 4
3.1.2. Server reboots resulting in a confused lease 3.1.2. Server Reboots Resulting in a Confused Lease
situation . . . . . . . . . . . . . . . . . . . . . . 5 Situation . . . . . . . . . . . . . . . . . . . . . . 5
3.1.3. Client complexity issues . . . . . . . . . . . . . . 6 3.1.3. Client Complexity Issues . . . . . . . . . . . . . . 6
3.2. Sources of Protocol difficulties . . . . . . . . . . . . 7 3.2. Sources of Protocol Difficulties . . . . . . . . . . . . 7
3.2.1. Issues with nfs_client_id4 generation and use . . . . 7 3.2.1. Issues with nfs_client_id4 Generation and Use . . . . 7
3.2.2. Issues with lease proliferation . . . . . . . . . . . 9 3.2.2. Issues with Lease Proliferation . . . . . . . . . . . 9
4. Issues to be resolved in NFSv4.0 . . . . . . . . . . . . . . 10 4. Issues Requiring Resolution in NFSv4.0 . . . . . . . . . . . 10
4.1. Possible changes to nfs_client_id4 client-string . . . . 10 4.1. Changes to nfs_client_id4 Client-string . . . . . . . . . 10
4.2. Possible changes to handle differing nfs_client_id4 4.2. Changes to Handle Differing nfs_client_id4 String Values 11
string values . . . . . . . . . . . . . . . . . . . . . . 11 4.3. Potential Changes to Add a New Operation . . . . . . . . 11
4.3. Possible changes to add a new operation . . . . . . . . . 12 4.4. Other Issues Within Migration-state Sections . . . . . . 12
4.4. Other issues within migration-state sections . . . . . . 12 4.5. Issues Within Other Sections . . . . . . . . . . . . . . 12
4.5. Issues within other sections . . . . . . . . . . . . . . 12 5. Resolution of NFSv4.0 Protocol Difficulties . . . . . . . . . 13
5. Proposed resolution of NFSv4.0 protocol difficulties . . . . 13 5.1. Changes Regarding nfs_client_id4 Client-string . . . . . 13
5.1. Proposed changes: nfs_client_id4 client-string . . . . . 13 5.2. Changes Regarding Merged (vs. Synchronized) Leases . . . 14
5.2. Proposed changes: merged (vs. synchronized) leases . . . 14 5.3. Other Changes to Migration-state Sections . . . . . . . . 15
5.3. Other proposed changes to migration-state sections . . . 15 5.3.1. Changes Regarding Client ID Migration . . . . . . . . 15
5.3.1. Proposed changes: Client ID migration . . . . . . . . 15 5.3.2. Changes Regarding Callback Re-establishment . . . . . 16
5.3.2. Proposed changes: Callback re-establishment . . . . . 16 5.3.3. NFS4ERR_LEASE_MOVED Rework . . . . . . . . . . . . . 16
5.3.3. Proposed changes: NFS4ERR_LEASE_MOVED rework . . . . 17 5.4. Changes to Other Sections . . . . . . . . . . . . . . . . 17
5.4. Proposed changes to other sections . . . . . . . . . . . 17 5.4.1. Callback Update . . . . . . . . . . . . . . . . . . . 17
5.4.1. Proposed changes: callback update . . . . . . . . . . 17 5.4.2. clientid4 Handling . . . . . . . . . . . . . . . . . 17
5.4.2. Proposed changes: clientid4 handling . . . . . . . . 17 5.4.3. Handling of NFS4ERR_CLID_INUSE . . . . . . . . . . . 19
5.4.3. Proposed changes: NFS4ERR_CLID_INUSE . . . . . . . . 19
6. Issues for NFSv4.1 . . . . . . . . . . . . . . . . . . . . . 20 6. Issues for NFSv4.1 . . . . . . . . . . . . . . . . . . . . . 20
6.1. Addressing state merger in NFSv4.1 . . . . . . . . . . . 20 6.1. Addressing state merger in NFSv4.1 . . . . . . . . . . . 20
6.2. Addressing pNFS relationship with migration . . . . . . . 21 6.2. Addressing pNFS relationship with migration . . . . . . . 21
6.3. Addressing server owner changes in NFSv4.1 . . . . . . . 21 6.3. Addressing server owner changes in NFSv4.1 . . . . . . . 21
7. Security Considerations . . . . . . . . . . . . . . . . . . . 23 7. Security Considerations . . . . . . . . . . . . . . . . . . . 22
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23
10.1. Normative References . . . . . . . . . . . . . . . . . . 23 10.1. Normative References . . . . . . . . . . . . . . . . . . 23
10.2. Informative References . . . . . . . . . . . . . . . . . 24 10.2. Informative References . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction 1. Introduction
This document is in the informational category, and while the facts This document is in the informational category, and while the facts
it reports may have normative implications, any such normative it reports may have normative implications, any such normative
significance reflects the readers' preferences. For example, we may significance reflects the readers' preferences. For example, we may
report that the reboot of a client with migrated state results in report that the reboot of a client with migrated state results in
state not being promptly cleared and that this will prevent granting state not being promptly cleared and that this will prevent granting
of conflicting lock requests at least for the lease time, which is a of conflicting lock requests at least for the lease time, which is a
skipping to change at page 3, line 43 skipping to change at page 3, line 41
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
In the context of this informational document, these normative In the context of this informational document, these normative
keywords will always occur in the context of a quotation, most often keywords will always occur in the context of a quotation, most often
direct but sometimes indirect. The context will make it clear direct but sometimes indirect. The context will make it clear
whether the quotation is from: whether the quotation is from:
o The current definitive definition of the NFSv4.0 protocol o The previously current definitive definition of the NFSv4.0
[RFC7530]. protocol [RFC7530].
o The current definitive definition of the NFSv4.1 protocol o The current definitive definition of the NFSv4.1 protocol
[RFC5661]. [RFC5661].
o A proposed or possible text to serve as a replacement for the o A proposed or possible text to serve as a replacement for the
current definitive document text. Sometimes, a number of possible current or previous definitive document text. Sometimes, a number
alternative texts may be listed and benefits and detriments of of possible alternative texts may be listed and benefits and
each examined in turn. detriments of each examined in turn.
3. NFSv4.0 Implementation Experience 3. NFSv4.0 Implementation Experience
3.1. Implementation issues 3.1. Implementation Issues
Note that the examples below reflect current experience which arises Note that the examples below reflect current experience which arises
from clients implementing the recommendation to use different from clients implementing the recommendation to use different
nfs_client_id4 id strings for different server addresses, i.e. using nfs_client_id4 id strings for different server addresses, i.e. using
what is later referred to herein as the "non-uniform client-string what is later referred to herein as the "non-uniform client-string
approach." approach."
This is simply because that is the experience implementers have had. This is simply because that is the experience implementers have had.
The reader should not assume that in all cases, this practice is the The reader should not assume that in all cases, this practice is the
source of the difficulty. It may be so in some cases but clearly it source of the difficulty. It may be so in some cases but clearly it
is not in all cases. is not in all cases.
3.1.1. Failure to free migrated state on client reboot 3.1.1. Failure to Free Migrated State on Client Reboot
The following sort of situation has proved troublesome: The following sort of situation has proved troublesome:
o A client C establishes a clientid4 C1 with server ABC specifying o A client C establishes a clientid4 C1 with server ABC specifying
an nfs_client_id4 with id string value "C-ABC" and boot verifier an nfs_client_id4 with id string value "C-ABC" and boot verifier
0x111. 0x111.
o The client begins to access files in filesystem F on server ABC, o The client begins to access files in filesystem F on server ABC,
resulting in generating stateids S1, S2, etc. under the lease for resulting in generating stateids S1, S2, etc. under the lease for
clientid C1. It may also access files on other filesystems on the clientid C1. It may also access files on other filesystems on the
skipping to change at page 5, line 5 skipping to change at page 5, line 5
they are associated with a different client name and so lease they are associated with a different client name and so lease
expiration is the only way that they can be gotten rid of. expiration is the only way that they can be gotten rid of.
Note here that while it seems clear to us in this example that C-XYZ Note here that while it seems clear to us in this example that C-XYZ
and C-ABC are from the same client, the server has no way to and C-ABC are from the same client, the server has no way to
determine the structure of the "opaque" id string. In the protocol, determine the structure of the "opaque" id string. In the protocol,
it really is treated as opaque. Only the client knows which it really is treated as opaque. Only the client knows which
nfs_client_id4 values designate the same client on a different nfs_client_id4 values designate the same client on a different
server. server.
3.1.2. Server reboots resulting in a confused lease situation 3.1.2. Server Reboots Resulting in a Confused Lease Situation
Further problems arise from scenarios like the following. Further problems arise from scenarios like the following.
o Client C talks to server ABC using an nfs_client_id4 id string o Client C talks to server ABC using an nfs_client_id4 id string
such as "C-ABC" and a boot verifier v1. As a result, a lease with such as "C-ABC" and a boot verifier v1. As a result, a lease with
clientid4 c.i is established: {v1, "C-ABC", c.i}. clientid4 c.i is established: {v1, "C-ABC", c.i}.
o fs_a1 migrates from server ABC to server XYZ along with its state. o fs_a1 migrates from server ABC to server XYZ along with its state.
Now server XYZ also has a lease: {v1, "C-ABC", c.i}. Now server XYZ also has a lease: {v1, "C-ABC", c.i}.
skipping to change at page 6, line 17 skipping to change at page 6, line 17
While this would be a reasonable patch for an isolated protocol While this would be a reasonable patch for an isolated protocol
weakness, interoperable clients and servers would require that the weakness, interoperable clients and servers would require that the
protocol truly be updated to allow such a situation, specifically protocol truly be updated to allow such a situation, specifically
that of multiple clientid4's with the same nfs_client_id4 value. The that of multiple clientid4's with the same nfs_client_id4 value. The
protocol is currently designed and implemented assuming this cannot protocol is currently designed and implemented assuming this cannot
happen. We need to either prevent the situation from happening, or happen. We need to either prevent the situation from happening, or
fully adapt to the possibilities which can arise. See Section 4 for fully adapt to the possibilities which can arise. See Section 4 for
a discussion of such issues. a discussion of such issues.
3.1.3. Client complexity issues 3.1.3. Client Complexity Issues
Consider the following situation: Consider the following situation:
o There are a set of clients C1 through Cn accessing servers S1 o There are a set of clients C1 through Cn accessing servers S1
through Sm. Each server manages some significant number of through Sm. Each server manages some significant number of
filesystems with the filesystem count L being significantly filesystems with the filesystem count L being significantly
greater than m. greater than m.
o Each client Cx will access a subset of the servers and so will o Each client Cx will access a subset of the servers and so will
have up to m clientids, which we will call Cxy for server Sy. have up to m clientids, which we will call Cxy for server Sy.
skipping to change at page 7, line 42 skipping to change at page 7, line 42
o Before doing an operation that can result in a stateid, the client o Before doing an operation that can result in a stateid, the client
must either find a "state blob" based on fsid or create a new one, must either find a "state blob" based on fsid or create a new one,
possibly with a new clientid4. possibly with a new clientid4.
o There may be multiple clientid4's all connected to the same server o There may be multiple clientid4's all connected to the same server
and using the same nfs_clientid4. and using the same nfs_clientid4.
This sort of additional client complexity is troublesome and needs to This sort of additional client complexity is troublesome and needs to
be eliminated. be eliminated.
3.2. Sources of Protocol difficulties 3.2. Sources of Protocol Difficulties
3.2.1. Issues with nfs_client_id4 generation and use 3.2.1. Issues with nfs_client_id4 Generation and Use
In the current definitive definition of the NFSv4.0 protocol, In [RFC7530], the section entitled "Client ID" says:
[RFC7530], the section entitled "Client ID" says:
The second field, id is a variable length string that uniquely The second field, id is a variable length string that uniquely
defines the client. defines the client.
There are two possible interpretations of the phrase "uniquely There are two possible interpretations of the phrase "uniquely
defines" in the above: defines" in the above:
o The relation between strings and clients is a function from such o The relation between strings and clients is a function from such
strings to clients so that each string designates a single client. strings to clients so that each string designates a single client.
skipping to change at page 9, line 6 skipping to change at page 8, line 52
different servers is for the same client or not, and this different servers is for the same client or not, and this
distinction becomes very important. distinction becomes very important.
Given the need for the server to be aware of client identity with Given the need for the server to be aware of client identity with
regard to migrated state, either client-string construction rules regard to migrated state, either client-string construction rules
will have to change or there will be a need to get around current will have to change or there will be a need to get around current
issues, or perhaps a combination of these two will be required. issues, or perhaps a combination of these two will be required.
Later sections will examine the options and propose a solution. Later sections will examine the options and propose a solution.
One consideration that may indicate that this cannot remain exactly One consideration that may indicate that this cannot remain exactly
as it is today has to do with the fact that the current explanation as it has been derives from the fact that the current explanation for
for this behavior is not correct. In the current definitive this behavior is not correct. In [RFC7530], the section entitled
definition of the NFSv4.0 protocol [RFC7530], the section entitled
"Client ID" says: "Client ID" says:
The reason is that it may not be possible for the client to tell The reason is that it may not be possible for the client to tell
if the same server is listening on multiple network addresses. If if the same server is listening on multiple network addresses. If
the client issues SETCLIENTID with the same id string to each the client issues SETCLIENTID with the same id string to each
network address of such a server, the server will think it is the network address of such a server, the server will think it is the
same client, and each successive SETCLIENTID will cause the server same client, and each successive SETCLIENTID will cause the server
to begin the process of removing the client's previous leased to begin the process of removing the client's previous leased
state. state.
skipping to change at page 9, line 39 skipping to change at page 9, line 35
The situation of multiple SETCLIENTID requests received by a server The situation of multiple SETCLIENTID requests received by a server
on multiple network addresses is exactly the same, from the protocol on multiple network addresses is exactly the same, from the protocol
design point of view, as when multiple (i.e. duplicate) SETCLIENTID design point of view, as when multiple (i.e. duplicate) SETCLIENTID
requests are received by the server on a single network address. The requests are received by the server on a single network address. The
same protocol mechanisms that prevent erroneous state deletion in the same protocol mechanisms that prevent erroneous state deletion in the
latter case prevent it in the former case. There is no reason for latter case prevent it in the former case. There is no reason for
special handling of the multiple-network-appearance case, in this special handling of the multiple-network-appearance case, in this
regard. regard.
3.2.2. Issues with lease proliferation 3.2.2. Issues with Lease Proliferation
It is often felt that this is a consequence of the client-string It is often felt that this is a consequence of the client-string
construction issues, and it is certainly the case that the two are construction issues, and it is certainly the case that the two are
closely connected in that non-uniform client-strings make it closely connected in that non-uniform client-strings make it
impossible for the server to appropriately combine leases from the impossible for the server to appropriately combine leases from the
same client. same client.
However, even where the server could combine leases from the same However, even where the server could combine leases from the same
client, it needs to be clear how and when it will do so, so that the client, it needs to be clear how and when it will do so, so that the
client will be prepared. These issues will have to be addressed at client will be prepared. These issues will have to be addressed at
various places in the spec. various places in the protocol specification.
This could be enough only if we are prepared to do away with the This could be enough only if we are prepared to do away with the
"should" recommending non-uniform client-strings and replace it with "should" recommending non-uniform client-strings and replace it with
a "should not" or even a "SHOULD NOT". Current client implementation a "should not" or even a "SHOULD NOT". Current client implementation
patterns make this an unpalatable choice for use as a general patterns make this an unpalatable choice for use as a general
solution, but it is reasonable to "RECOMMEND" this choice for a well- solution, but it is reasonable to "RECOMMEND" this choice for a well-
defined subset of clients. One alternative would be to create a way defined subset of clients. One alternative would be to create a way
for the server to infer from client behavior which leases are held by for the server to infer from client behavior which leases are held by
the same client and use this information to do appropriate lease the same client and use this information to do appropriate lease
mergers. Prototyping and detailed specification work has shown that mergers. Prototyping and detailed specification work has shown that
skipping to change at page 10, line 26 skipping to change at page 10, line 21
for clients supporting the migration feature. for clients supporting the migration feature.
Because of the discussion of client-string construction in [RFC7530], Because of the discussion of client-string construction in [RFC7530],
most existing clients implement the non-uniform client-string most existing clients implement the non-uniform client-string
approach. As a result, existing servers may not have been tested approach. As a result, existing servers may not have been tested
with clients implementing uniform client-strings. As a consequence, with clients implementing uniform client-strings. As a consequence,
care must be taken to preserve interoperability between UCS-capable care must be taken to preserve interoperability between UCS-capable
clients and servers that don't tolerate uniform client strings for clients and servers that don't tolerate uniform client strings for
one reason or another. one reason or another.
4. Issues to be resolved in NFSv4.0 4. Issues Requiring Resolution in NFSv4.0
4.1. Possible changes to nfs_client_id4 client-string 4.1. Changes to nfs_client_id4 Client-string
The fact that the reason given in client-string-BP3 is not valid The fact that the reason given in client-string-BP3 is not valid
makes the existing "should" insupportable. We can't either makes the existing "should" insupportable. We can't either
o Keep a reason we know is invalid. o Keep a reason we know is invalid.
o Keep saying "should" without giving a reason. o Keep saying "should" without giving a reason.
What are often presented as reasons that motivate use of the non- What are often presented as reasons that motivate use of the non-
uniform approach always turn out to be cases in which, if the uniform uniform approach always turn out to be cases in which, if the uniform
approach were used, the server will treat a client which accesses approach were used, the server will treat a client which accesses
that server via two different IP addresses as part of a single that server via two different IP addresses as part of a single
client, as it in fact is. This may be disconcerting to a client client, as it in fact is. This may be disconcerting to a client
unaware that the two IP addresses connect to the same server. This unaware that the two IP addresses connect to the same server. This
is not a reason to use the non-uniform approach but is better thought is not a reason to use the non-uniform approach but is better thought
of as an illustration of the fact that those using the uniform of as an illustration of the fact that those using the uniform
approach need to be aware of the possibility of server trunking and approach need to be aware of the possibility of server trunking and
its potential effect on server behavior. its potential effect on server behavior.
If it is possible to reliably infer the existence of trunking of Since it is possible to reliably infer the existence of trunking of
server IP addresses from observed server behavior, use of the uniform server IP addresses from observed server behavior, use of the uniform
approach would be more desirable, although compatibility issues would approach is more desirable, although compatibility issues need to be
have to be dealt with. dealt with.
An alternative to having the client infer the existence of trunking An alternative to having the client infer the existence of trunking
of IP server addresses, is to make this information available to the of IP server addresses, is to make this information available to the
client directly. See Section 4.3 for details. client directly. See Section 4.3 for details.
It is always possible that a valid new reason will be found, but so It is always possible that a valid new reason will be found, but so
far none has been proposed. Given the history, the burden of proof far none has been proposed. Given the history, the burden of proof
should be on those asserting the validity of a proposed new reason. was on those asserting the validity of a proposed new reason.
So we will assume for now that the "should" will have to go. The So we will assume that the "should" needs to go. The question was
question is what to replace it with. what to replace it with.
o We can't say "MUST NOT", despite the problems this raises for o We can't say "MUST NOT", despite the problems this raises for
migration since this is pretty late in the day for such a change. migration since this is pretty late in the day for such a change.
Many currently operating clients obey the existing "should". Many currently operating clients obey the existing "should".
Similar considerations would apply for "SHOULD NOT" or "should Similar considerations would apply for "SHOULD NOT" or "should
not". not".
o Dropping client-string-BP3 entirely is a possibility but, given o Dropping client-string-BP3 entirely is a possibility but, given
the context and history, it would just be a confusing version of the context and history, it would just be a confusing version of
"SHOULD NOT". "SHOULD NOT".
o Using "MAY" would clearly specify that both ways of doing this are o Using "MAY" would clearly specify that both ways of doing this are
valid choices for clients and that servers will have to deal with valid choices for clients and that servers will have to deal with
clients that make either choice. clients that make either choice.
o This might be modified by a "SHOULD" (or even a "MUST") for o This might be modified by a "SHOULD" (or even a "MUST") for
particular groups of clients. particular groups of clients.
o There will have to be some text explaining why a client might make o There has to be some text explaining why a client might make
either choice but, except for the particular cases referred to either choice but, except for the particular cases referred to
above, we will have to make sure that it is truly descriptive, and above, it will have to made sure that it is truly descriptive, and
not slanted in either direction. not slanted in either direction.
4.2. Possible changes to handle differing nfs_client_id4 string values 4.2. Changes to Handle Differing nfs_client_id4 String Values
Given the difficulties caused by having different nfs_client_id4 Given the difficulties caused by having different nfs_client_id4
client-string values for the same client, we have two choices: client-string values for the same client, we had two choices:
o Deprecate the existing treatment and basically say the client is o Deprecating the existing treatment and basically saying the client
on its own doing migration, if it follows it. is on its own doing migration, if it follows it.
o Introduce a way of having the client provide client identity o Introducing a way of having the client provide client identity
information to the server, if it can be done compatibly while information to the server, if it can be done compatibly while
staying within the bounds of v4.0. staying within the bounds of v4.0.
4.3. Possible changes to add a new operation 4.3. Potential Changes to Add a New Operation
It might be possible to return server-identity information to the It might be possible to return server-identity information to the
client, just as is done in NFSv4.1 by the response to the EXCHANGE_ID client, just as is done in NFSv4.1 by the response to the EXCHANGE_ID
operation. This could be done by a SETCLIENTID_PLUS optional operation. This could be done by a SETCLIENTID_PLUS optional
operation, which acts like SETCLIENTID, except that it returns server operation, which acts like SETCLIENTID, except that it returns server
identity information. Such information could be used by clients, identity information. Such information could be used by clients,
making it possible to for them to be aware of server trunking making it possible to for them to be aware of server trunking
relationships, rather than having to infer them from server behavior. relationships, rather than having to infer them from server behavior.
It has been generally thought that protocol extensions such as this It has been generally thought that protocol extensions such as this
are not appropriate in bis documents and other documents updating are not appropriate in bis documents and other documents updating
NFSv4 protocol definition RFC's. However, [NFSv4-vers] discusses NFSv4 protocol definition RFC's. However, [NFSv4-vers] discusses
means by which protocol extensions, similar to those allowed between means by which protocol extensions, similar to those allowed between
minor versions, can be used to correct protocol mistakes. minor versions, could be used to correct protocol mistakes.
A decision to adopt this approach would require waiting for A decision to adopt this approach would require waiting for
[NFSv4-vers] to become a Proposed Standard. In view of the time [NFSv4-vers] to become a Proposed Standard. In view of the time
necessary for that to happen, this approach is not expected to be necessary for that to happen, this approach was not available in an
adopted in an RFC updating [RFC7530], such as [migr-v4.0-update]. RFC updating [RFC7530], such as [RFC7931]. Still, it is worth
Still, it is worth keeping in mind, if implementers have difficulties keeping in mind, if implementers have difficulties inferring trunking
inferring trunking relationships using the techniques discussed relationships using the techniques discussed there.
there.
4.4. Other issues within migration-state sections 4.4. Other Issues Within Migration-state Sections
There are a number of issues where the existing text is unclear and/ There are a number of issues where the existing text is unclear and/
or wrong and needs to be fixed in some way. or wrong and needs to be fixed in some way.
o Lack of clarity in the discussion of moving clientids (as well as o Lack of clarity in the discussion of moving clientids (as well as
stateids) as part of moving state for migration. stateids) as part of moving state for migration.
o The discussion of synchronized leases is wrong in that there is no o The discussion of synchronized leases is wrong in that there is no
way to determine (in the current spec) when leases are for the way to determine (in the current spec) when leases are for the
same client and also wrong in suggesting a benefit from leases same client and also wrong in suggesting a benefit from leases
synchronized at the point of transfer. What is needed is merger synchronized at the point of transfer. What is needed is merger
of leases, which is necessary to keep client complexity of leases, which is necessary to keep client complexity
requirements from getting out of hand. requirements from getting out of hand.
o Lack of clarity in the discussion of LEASE_MOVED handling, o Lack of clarity in the discussion of LEASE_MOVED handling,
including failure to fully address situations in which transparent including failure to fully address situations in which transparent
state migration did not occur. state migration did not occur.
4.5. Issues within other sections 4.5. Issues Within Other Sections
There are a number of cases in which certain sections, not There are a number of cases in which certain sections, not
specifically related to migration, require additional clarification. specifically related to migration, require additional clarification.
This is generally because text that is clear in a context in which This is generally because text that is clear in a context in which
leases and clientids are created in one place and live there forever leases and clientids are created in one place and live there forever
may need further refinement in the more dynamic environment that may need further refinement in the more dynamic environment that
arises as part of migration. arises as part of migration.
Some examples: Some examples:
skipping to change at page 13, line 26 skipping to change at page 13, line 18
needs to be clarified. In particular, the issue of how the client needs to be clarified. In particular, the issue of how the client
adapts to the presumably independent and uncoordinated clientid4 adapts to the presumably independent and uncoordinated clientid4
sets needs to be clearly addressed sets needs to be clearly addressed
o Statements regarding handling of invalid clientid4's need to be o Statements regarding handling of invalid clientid4's need to be
clarified and/or refined in light of the possibilities that arise clarified and/or refined in light of the possibilities that arise
due to lease motion and merger. due to lease motion and merger.
o Confusion and lack of clarity about NFS4ERR_CLID_INUSE. o Confusion and lack of clarity about NFS4ERR_CLID_INUSE.
5. Proposed resolution of NFSv4.0 protocol difficulties 5. Resolution of NFSv4.0 Protocol Difficulties
This section lists the changes which we believe are necessary to This section lists the changes that were necessary to resolve the
resolve the difficulties mentioned above. Such changes, along with difficulties mentioned above. Such changes, along with other
other clarifications found to be desirable during drafting and review clarifications found to be desirable during drafting and review are
are contained in [migr-v4.0-update]. contained in [RFC7931].
5.1. Proposed changes: nfs_client_id4 client-string 5.1. Changes Regarding nfs_client_id4 Client-string
We propose replacing client-string-BP3 with the following text: It was decided to replace client-string-BP3 with the following text:
The string MAY be different for each server network address that The string MAY be different for each server network address that
the client accesses, rather than common to all server network the client accesses, rather than common to all server network
addresses. addresses.
In addition, given the importance of the issue of client identity and In addition, given the importance of the issue of client identity and
the fact that both client string-approaches are to be considered the fact that both client string-approaches are to be considered
valid, a greatly expanded treatment of client identity desirable. It valid, a greatly expanded treatment of client identity was desirable.
should have the following major elements. It had the following major elements.
o It should fully describe the consequences of making the string o Fully describing the consequences of making the string different
different for each network address (the non-uniform client-string for each network address (the non-uniform client-string approach)
approach) and of making it the same for all network addresses (the and of making it the same for all network addresses (the uniform
uniform client string approach). client string approach).
o It should give helpful guidance about the factors that might o Giving helpful guidance about the factors that might affect client
affect client implementation choice between these approaches. implementation choice between these approaches.
o It should describe the compatibility issues that might cause o Describing the compatibility issues that might cause servers to be
servers to be incompatible with the uniform approach and give incompatible with the uniform approach and give guidance about
guidance about dealing with these. dealing with these.
o It should describe how a client using the uniform approach might o Describing how a client using the uniform approach might use
use server behavior to determine server address trunking patterns. server behavior to determine server address trunking patterns.
o It should present a clearer and more complete set of o Presenting a clearer and more complete set of recommendations to
recommendations to guide client string construction. guide client string construction.
5.2. Proposed changes: merged (vs. synchronized) leases 5.2. Changes Regarding Merged (vs. Synchronized) Leases
In the current definitive definition of the NFSv4.0 protocol, In [RFC7530], the section entitled "Migration and State" says:
[RFC7530], the section entitled "Migration and State" says:
As part of the transfer of information between servers, leases As part of the transfer of information between servers, leases
would be transferred as well. The leases being transferred to the would be transferred as well. The leases being transferred to the
new server will typically have a different expiration time from new server will typically have a different expiration time from
those for the same client, previously on the old server. To those for the same client, previously on the old server. To
maintain the property that all leases on a given server for a maintain the property that all leases on a given server for a
given client expire at the same time, the server should advance given client expire at the same time, the server should advance
the expiration time to the later of the leases being transferred the expiration time to the later of the leases being transferred
or the leases already present. This allows the client to maintain or the leases already present. This allows the client to maintain
lease renewal of both classes without special effort: lease renewal of both classes without special effort:
There are a number of problems with this and any resolution of our There are a number of problems with this and any resolution of our
difficulties must address them somehow. difficulties must address them somehow.
o The current v4.0 spec recommends that the client make it o [RFC7530] recommends that the client make it essentially
essentially impossible to determine when two leases are from "the impossible to determine when two leases are from "the same
same client". client".
o It is not appropriate to speak of "maintain[ing] the property that o It is not appropriate to speak of "maintain[ing] the property that
all leases on a given server for a given client expire at the same all leases on a given server for a given client expire at the same
time", since this is not a property that holds even in the absence time", since this is not a property that holds even in the absence
of migration. A server listening on multiple network addresses of migration. A server listening on multiple network addresses
may have the same client appear as multiple clients with no way to may have the same client appear as multiple clients with no way to
recognize the client as the same. recognize the client as the same.
o Even if the client identity issue could be resolved, advancing the o Even if the client identity issue could be resolved, advancing the
lease time at the point of migration would not maintain the lease time at the point of migration would not maintain the
skipping to change at page 15, line 38 skipping to change at page 15, line 26
If servers obey the SHOULD and clients choose to adopt the uniform id If servers obey the SHOULD and clients choose to adopt the uniform id
approach, having more than a single lease for a given client-server approach, having more than a single lease for a given client-server
pair will be a transient situation, cleaned up as part of adapting to pair will be a transient situation, cleaned up as part of adapting to
use of migrated state. use of migrated state.
Since clients and servers will be a mixture of old and new and Since clients and servers will be a mixture of old and new and
because nothing is a MUST we have to ensure that no combination will because nothing is a MUST we have to ensure that no combination will
show worse behavior than is exhibited by current (i.e. old) clients show worse behavior than is exhibited by current (i.e. old) clients
and servers. and servers.
5.3. Other proposed changes to migration-state sections 5.3. Other Changes to Migration-state Sections
5.3.1. Proposed changes: Client ID migration 5.3.1. Changes Regarding Client ID Migration
In the current definitive definition of the NFSv4.0 protocol In [RFC7530], the section entitled "Migration and State" says:
[RFC7530], the section entitled "Migration and State" says:
In the case of migration, the servers involved in the migration of In the case of migration, the servers involved in the migration of
a filesystem SHOULD transfer all server state from the original to a filesystem SHOULD transfer all server state from the original to
the new server. This must be done in a way that is transparent to the new server. This must be done in a way that is transparent to
the client. This state transfer will ease the client's transition the client. This state transfer will ease the client's transition
when a filesystem migration occurs. If the servers are successful when a filesystem migration occurs. If the servers are successful
in transferring all state, the client will continue to use in transferring all state, the client will continue to use
stateids assigned by the original server. Therefore the new stateids assigned by the original server. Therefore the new
server must recognize these stateids as valid. This holds true server must recognize these stateids as valid. This holds true
for the client ID as well. Since responsibility for an entire for the client ID as well. Since responsibility for an entire
skipping to change at page 16, line 34 skipping to change at page 16, line 24
We have decided that it is best to address this issue as follows: We have decided that it is best to address this issue as follows:
o Make it clear that both clientid4 and nfs_client_id4 (including o Make it clear that both clientid4 and nfs_client_id4 (including
both id string and boot verifier) are to be transferred. both id string and boot verifier) are to be transferred.
o Indicate that the initial transfer will result in the same o Indicate that the initial transfer will result in the same
clientid4 after transfer but this is not guaranteed since there clientid4 after transfer but this is not guaranteed since there
may conflict with an existing clientid4 on the destination server may conflict with an existing clientid4 on the destination server
and because lease merger can result in a change of the clientid4. and because lease merger can result in a change of the clientid4.
5.3.2. Proposed changes: Callback re-establishment 5.3.2. Changes Regarding Callback Re-establishment
In the current definitive definition of the NFSv4.0 protocol In [RFC7530], the section entitled "Migration and State" says:
[RFC7530], the section entitled "Migration and State" says:
A client SHOULD re-establish new callback information with the new A client SHOULD re-establish new callback information with the new
server as soon as possible, according to sequences described in server as soon as possible, according to sequences described in
sections "Operation 35: SETCLIENTID - Negotiate Client ID" and sections "Operation 35: SETCLIENTID - Negotiate Client ID" and
"Operation 36: SETCLIENTID_CONFIRM - Confirm Client ID". This "Operation 36: SETCLIENTID_CONFIRM - Confirm Client ID". This
ensures that server operations are not blocked by the inability to ensures that server operations are not blocked by the inability to
recall delegations. recall delegations.
The above will need to be fixed to reflect the possibility of merging The above will need to be fixed to reflect the possibility of merging
of leases, of leases,
5.3.3. Proposed changes: NFS4ERR_LEASE_MOVED rework 5.3.3. NFS4ERR_LEASE_MOVED Rework
In the current definitive definition of the NFSv4.0 protocol In [RFC7530], the section entitled "Notification of Migrated Lease"
[RFC7530], the section entitled "Notification of Migrated Lease"
says: says:
Upon receiving the NFS4ERR_LEASE_MOVED error, a client that Upon receiving the NFS4ERR_LEASE_MOVED error, a client that
supports filesystem migration MUST probe all filesystems from that supports filesystem migration MUST probe all filesystems from that
server on which it holds open state. Once the client has server on which it holds open state. Once the client has
successfully probed all those filesystems which are migrated, the successfully probed all those filesystems which are migrated, the
server MUST resume normal handling of stateful requests from that server MUST resume normal handling of stateful requests from that
client. client.
There is a lack of clarity that is prompted by ambiguity about what There is a lack of clarity that is prompted by ambiguity about what
exactly probing is and what the interlock between client and server exactly probing is and what the interlock between client and server
must be. This has led to some worry about the scalability of the must be. This has led to some worry about the scalability of the
probing process, and although the time required does scale linearly probing process, and although the time required does scale linearly
with the number of filesystems that the client may have state for with the number of filesystems that the client may have state for
with respect to a given server, the actual process can be done with respect to a given server, the actual process can be done
efficiently. efficiently.
To address these issues we propose rewriting the above to be more To address these issues, the text above had to be rewritten to be
clear and to give suggestions about how to do the required scanning more clear and to give suggestions about how to do the required
efficiently. scanning efficiently.
5.4. Proposed changes to other sections 5.4. Changes to Other Sections
5.4.1. Proposed changes: callback update 5.4.1. Callback Update
Some changes are necessary to reduce confusion about the process of Some changes are necessary to reduce confusion about the process of
callback information update and in particular to make it clear that callback information update and in particular to make it clear that
no state is freed as a result: no state is freed as a result:
o Make it clear that after migration there are confirmed entries for o Make it clear that after migration there are confirmed entries for
transferred clientid4/nfs_client_id4 pairs. transferred clientid4/nfs_client_id4 pairs.
o Be explicit in the sections headed "otherwise," in the o Be explicit in the sections headed "otherwise," in the
descriptions of SETCLIENTID and SETCLIENTID_CONFIRM, that these descriptions of SETCLIENTID and SETCLIENTID_CONFIRM, that these
don't apply in the cases we are concerned about. don't apply in the cases we are concerned about.
5.4.2. Proposed changes: clientid4 handling 5.4.2. clientid4 Handling
To address both of the clientid4-related issues mentioned in To address both of the clientid4-related issues mentioned in
Section 4.5, we propose replacing the last three paragraphs of the Section 4.5, it was necessary to replace the last three paragraphs of
section entitled "Client ID" with the following: the section entitled "Client ID" with the following:
Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has Once a SETCLIENTID and SETCLIENTID_CONFIRM sequence has
successfully completed, the client uses the shorthand client successfully completed, the client uses the shorthand client
identifier, of type clientid4, instead of the longer and less identifier, of type clientid4, instead of the longer and less
compact nfs_client_id4 structure. This shorthand client compact nfs_client_id4 structure. This shorthand client
identifier (a client ID) is assigned by the server and should be identifier (a client ID) is assigned by the server and should be
chosen so that it will not conflict with a client ID previously chosen so that it will not conflict with a client ID previously
assigned by same server. This applies across server restarts or assigned by same server. This applies across server restarts or
reboots. reboots.
skipping to change at page 19, line 26 skipping to change at page 19, line 11
receives a NFS4ERR_STALE_STATEID error using a stateid derived receives a NFS4ERR_STALE_STATEID error using a stateid derived
from its current clientid4, since this indicates a situation, such from its current clientid4, since this indicates a situation, such
as server reboot which has invalidated the existing clientid4 and as server reboot which has invalidated the existing clientid4 and
associated stateids (see the section entitled "lock-owner" for associated stateids (see the section entitled "lock-owner" for
details). details).
See the detailed descriptions of SETCLIENTID and See the detailed descriptions of SETCLIENTID and
SETCLIENTID_CONFIRM for a complete specification of the SETCLIENTID_CONFIRM for a complete specification of the
operations. operations.
5.4.3. Proposed changes: NFS4ERR_CLID_INUSE 5.4.3. Handling of NFS4ERR_CLID_INUSE
It appears to be the intention that only a single principal be used It appears to be the intention that only a single principal be used
for client establishment between any client-server pair. However: for client establishment between any client-server pair. However:
o There is no explicit statement to this effect. o There is no explicit statement to this effect.
o The error that indicates a principal conflict has a name which o The error that indicates a principal conflict has a name which
does not clarify this issue: NFS4ERR_CLID_INUSE. does not clarify this issue: NFS4ERR_CLID_INUSE.
o The definition of the error is also not very helpful: "The o The definition of the error is also not very helpful: "The
skipping to change at page 24, line 5 skipping to change at page 23, line 43
[RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed., [RFC5661] Shepler, S., Ed., Eisler, M., Ed., and D. Noveck, Ed.,
"Network File System (NFS) Version 4 Minor Version 1 "Network File System (NFS) Version 4 Minor Version 1
Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010, Protocol", RFC 5661, DOI 10.17487/RFC5661, January 2010,
<http://www.rfc-editor.org/info/rfc5661>. <http://www.rfc-editor.org/info/rfc5661>.
[RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System [RFC7530] Haynes, T., Ed. and D. Noveck, Ed., "Network File System
(NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530, (NFS) Version 4 Protocol", RFC 7530, DOI 10.17487/RFC7530,
March 2015, <http://www.rfc-editor.org/info/rfc7530>. March 2015, <http://www.rfc-editor.org/info/rfc7530>.
10.2. Informative References [RFC7931] Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker,
"NFSv4.0 Migration: Specification Update", RFC 7931,
[migr-v4.0-update] DOI 10.17487/RFC7931, July 2016,
Noveck, D., Ed., Shivam, P., Lever, C., and B. Baker, <http://www.rfc-editor.org/info/rfc7931>.
"NFSv4.0 migration: Specification Update", 2016,
<http://www.ietf.org/id/
draft-ietf-nfsv4-rfc3530-migration-update-08.txt>.
Work in progress. 10.2. Informative References
[NFSv4-vers] [NFSv4-vers]
Noveck, D., "NFSv4 Version Management", 2016, Noveck, D., "NFSv4 Version Management", July 2016,
<http://www.ietf.org/id/ <http://www.ietf.org/id/
draft-ietf-nfsv4-versioning-03.txt>. draft-ietf-nfsv4-versioning-05.txt>.
Work in progress. Work in progress.
Authors' Addresses Authors' Addresses
David Noveck (editor) David Noveck (editor)
Hewlett Packard Enterprise Hewlett Packard Enterprise
165 Dascomb Road 165 Dascomb Road
Andover, MA 01810 Andover, MA 01810
US US
 End of changes. 67 change blocks. 
138 lines changed or deleted 126 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/