draft-ietf-nfsv4-migration-issues-15.txt   draft-ietf-nfsv4-migration-issues-16.txt 
NFSv4 D. Noveck, Ed. NFSv4 D. Noveck, Ed.
Internet-Draft NetApp Internet-Draft NetApp
Intended status: Informational P. Shivam Intended status: Informational P. Shivam
Expires: November 20, 2018 IBM Expires: February 22, 2019 IBM
C. Lever C. Lever
B. Baker B. Baker
ORACLE ORACLE
May 19, 2018 August 21, 2018
NFSv4 Migration and Trunking: Implementation and Specification Issues NFSv4 Migration and Trunking: Implementation and Specification Issues
draft-ietf-nfsv4-migration-issues-15 draft-ietf-nfsv4-migration-issues-16
Abstract Abstract
This document discusses a range of implementation and specification This document discusses a range of implementation and specification
issues concerning features related to the use of location-related issues concerning features related to the use of location-related
attributes in NFSv4. These include migration, which transfers attributes in NFSv4. These include migration, which transfers
responsibility for a file system from one server to another, and responsibility for a file system from one server to another, and
trunking which deals with the discovery and control of the set of trunking which deals with the discovery and control of the set of
server endpoints to use to access a file system. The focus of the server endpoints to use to access a file system. The focus of the
discussion, which relates to multiple minor versions, is on defining discussion, which relates to multiple minor versions, is on defining
skipping to change at page 1, line 42 skipping to change at page 1, line 42
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 20, 2018. This Internet-Draft will expire on February 22, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 22 skipping to change at page 2, line 22
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Language . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Language . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Requirements Language . . . . . . . . . . . . . . . . . . 4 2.1. Requirements Language . . . . . . . . . . . . . . . . . . 4
2.2. Use of Normative Terms . . . . . . . . . . . . . . . . . 4 2.2. Use of Normative Terms . . . . . . . . . . . . . . . . . 4
2.3. Terminology Used in this Document . . . . . . . . . . . . 5 2.3. Terminology Used in this Document . . . . . . . . . . . . 5
3. Issues that Apply to Multiple Versions and their Resolution . 7 3. Issues that Apply to Multiple Minor Versions and their
Resolution . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Issue Summary . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Issue Summary . . . . . . . . . . . . . . . . . . . . . . 7
3.2. Resolution of Multi-Version Issues . . . . . . . . . . . 8 3.2. Resolution of Multi-Version Issues . . . . . . . . . . . 8
3.2.1. Providing Trunking Discovery . . . . . . . . . . . . 9 3.2.1. Providing Trunking Discovery . . . . . . . . . . . . 9
3.2.2. Interaction of Trunking and Migration . . . . . . . . 10 3.2.2. Addressing Changes in Trunking Configuration . . . . 11
3.2.3. Dealing with Multiple Connection Types . . . . . . . 12 3.2.3. Interaction of Trunking and Migration . . . . . . . . 11
4. NFSv4.0 Issues . . . . . . . . . . . . . . . . . . . . . . . 13 3.2.4. Dealing with Multiple Connection Types . . . . . . . 13
4.1. Core NFSv4.0 Migration Issues . . . . . . . . . . . . . . 13 4. NFSv4.0 Issues . . . . . . . . . . . . . . . . . . . . . . . 14
4.1. Core NFSv4.0 Migration Issues . . . . . . . . . . . . . . 14
4.2. Resolution of Core Migration Protocol Difficulties in 4.2. Resolution of Core Migration Protocol Difficulties in
NFSv4.0 . . . . . . . . . . . . . . . . . . . . . . . . . 14 NFSv4.0 . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3. Additional NFSv4.0 Issues . . . . . . . . . . . . . . . . 15 4.3. Additional NFSv4.0 Issues . . . . . . . . . . . . . . . . 16
4.4. Resolution of Additional NFSv4.0 Issues . . . . . . . . . 15 4.4. Resolution of Additional NFSv4.0 Issues . . . . . . . . . 16
4.4.1. Resolution of NFSv4.0 Issues with Multiple Connection 4.4.1. Resolution of NFSv4.0 Issues with Multiple Connection
Types . . . . . . . . . . . . . . . . . . . . . . . . 17 Types . . . . . . . . . . . . . . . . . . . . . . . . 17
5. Issues for NFSv4.1 and Beyond . . . . . . . . . . . . . . . . 18 5. Issues for NFSv4.1 and Beyond . . . . . . . . . . . . . . . . 19
5.1. Issues to Address for NFSv4.1 . . . . . . . . . . . . . . 18 5.1. Trunking-focused Issues to Address for NFSv4.1 . . . . . 19
5.1.1. Addressing State Merger in NFSv4.1 . . . . . . . . . 19 5.1.1. Handling of Additional Information in
5.1.2. Addressing pNFS Relationship with Migration . . . . . 20 fs_locations_info . . . . . . . . . . . . . . . . . . 19
5.1.3. Addressing Server_owner Changes in NFSv4.1 . . . . . 20 5.1.2. Addressing Server_owner Changes in NFSv4.1 . . . . . 19
5.1.4. Addressing Confirmation Status of Migrated 5.2. Migration-related Issues to Address for NFSv4.1 . . . . . 21
Client IDs in NFSv4.1 . . . . . . . . . . . . . . . . 21 5.2.1. Addressing State Merger in NFSv4.1 . . . . . . . . . 22
5.1.5. Addressing Changes in Trunking Configuration . . . . 23 5.2.2. Addressing pNFS Relationship with Migration . . . . . 22
5.1.6. Addressing Session Migration in NFSv4.1 . . . . . . . 23 5.2.3. Addressing Confirmation Status of Migrated
5.1.7. Dealing with Multiple Connection Types in NFSv4.1 . . 23 Client IDs in NFSv4.1 . . . . . . . . . . . . . . . . 23
5.2. Possible Resolutions for NFSv4.1 Protocol Issues . . . . 24 5.2.4. Addressing Session Migration in NFSv4.1 . . . . . . . 24
5.2.1. Client ID Confirmation Issues . . . . . . . . . . . . 25 5.2.5. Dealing with Multiple Connection Types in NFSv4.1 . . 24
5.2.2. Dealing with Multiple Location Entries . . . . . . . 26 5.3. Possible Resolutions for NFSv4.1 Protocol Issues . . . . 26
5.2.3. Migration and pNFS . . . . . . . . . . . . . . . . . 28 5.3.1. Client ID Confirmation Issues . . . . . . . . . . . . 26
5.3. Defining Server Responsibilities for NFSv4.1 . . . . . . 29 5.3.2. Dealing with Multiple Location Entries . . . . . . . 27
5.3.1. Server Responsibilities in Effecting Transparent 5.3.3. Migration and pNFS . . . . . . . . . . . . . . . . . 29
State Migration . . . . . . . . . . . . . . . . . . . 29 5.4. Defining Server Responsibilities for NFSv4.1 Transitions 30
5.4.1. Server Responsibilities in Effecting Transparent
5.3.2. Synchronizing Session Transfer . . . . . . . . . . . 30 State Migration . . . . . . . . . . . . . . . . . . . 30
5.4. Defining Client Responsibilities for NFSv4.1 . . . . . . 33 5.4.2. Synchronizing Session Transfer . . . . . . . . . . . 32
5.4.1. Client Recovery from Migration Events . . . . . . . . 33 5.4.3. Server Issues Dealing with RECLAIM_COMPLETE . . . . . 34
5.4.2. The Migration Discovery Process . . . . . . . . . . . 35 5.5. Defining Client Responsibilities for NFSv4.1 Transitions 35
5.4.3. Overview of Client Response to NFS4ERR_MOVED . . . . 37 5.5.1. Client Recovery from Migration Events . . . . . . . . 35
5.4.4. Obtaining Access to Sessions and State after 5.5.2. The Migration Discovery Process . . . . . . . . . . . 38
Migration . . . . . . . . . . . . . . . . . . . . . . 39 5.5.3. Overview of Client Response to NFS4ERR_MOVED . . . . 39
5.4.5. Obtaining Access to Sessions and State after Network 5.5.4. Obtaining Access to Sessions and State after
Address Transfer . . . . . . . . . . . . . . . . . . 40 Migration . . . . . . . . . . . . . . . . . . . . . . 41
5.5. Resolution of NFSv4.1 Issues . . . . . . . . . . . . . . 41 5.5.5. Obtaining Access to Sessions and State after Network
5.6. Potential Protocol Extensions . . . . . . . . . . . . . . 43 Address Transfer . . . . . . . . . . . . . . . . . . 43
6. Evolution of Issue Handling . . . . . . . . . . . . . . . . . 44 5.6. Resolution of NFSv4.1 Issues . . . . . . . . . . . . . . 43
6.1. History of this Document . . . . . . . . . . . . . . . . 44 5.7. Potential Protocol Extensions . . . . . . . . . . . . . . 46
6.2. Further Work Needed . . . . . . . . . . . . . . . . . . . 46 6. Evolution of Issue Handling . . . . . . . . . . . . . . . . . 47
7. Security Considerations . . . . . . . . . . . . . . . . . . . 49 6.1. History of this Document . . . . . . . . . . . . . . . . 47
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 50 6.2. Further Work Needed . . . . . . . . . . . . . . . . . . . 48
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 50 7. Security Considerations . . . . . . . . . . . . . . . . . . . 52
9.1. Normative References . . . . . . . . . . . . . . . . . . 50 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 53
9.2. Informative References . . . . . . . . . . . . . . . . . 51 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 53
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 52 9.1. Normative References . . . . . . . . . . . . . . . . . . 53
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 52 9.2. Informative References . . . . . . . . . . . . . . . . . 54
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 54
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 55
1. Introduction 1. Introduction
This is an informational document that discusses a number of related This is an informational document that discusses a number of related
issues in multiple versions of NFSv4. issues in multiple minor versions of NFSv4 (Network File System
Version 4).
Many of these relate to the migration feature of NFSv4, which Many of these relate to the migration feature of NFSv4, which
provides for moving responsibility for a single filesystem from one provides for moving responsibility for a single filesystem from one
server to another, without disruption to clients. A number of server to another, without disruption to clients. A number of
problems in the specification of this feature in NFSv4.0 were problems in the specification of this feature in NFSv4.0 were
resolved by the publication of [RFC7931], which added trunking resolved by the publication of [RFC7931], which added trunking
detection to NFSV4.0. However, NFSv4.0 remains without an detection to NFSV4.0. However, NFSv4.0 remains without an
appropriate discussion of trunking discovery, which has many appropriate discussion of trunking discovery, which has many
important connections with migration. As a result, NFSv4.0 requires important connections with migration. As a result, NFSv4.0 requires
clarification of how the client is to respond to changes in the clarification of how the client is to respond to changes in the
skipping to change at page 4, line 25 skipping to change at page 4, line 31
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
2.2. Use of Normative Terms 2.2. Use of Normative Terms
This document, which deals with existing issues/problems in This document, which deals with existing issues/problems in
standards-track documents, is in the informational category, and standards-track documents, is in the informational category, and
while the facts it reports may have normative implications, any such while the facts it reports may have normative implications, any such
normative significance reflects the readers' preferences. For normative significance is left for the readers to determine. For
example, we may report that the existing definition of migration for example, we may report that the existing definition of migration for
NFSv4.1 does not properly describe how migrating state is to be NFSv4.1 does not properly describe how migrating state is to be
merged with existing state for the destination server. While it is merged with existing state for the destination server. While it is
to be expected that client and server implementers will judge this to to be expected that client and server implementers will judge this to
be a situation that it would be appropriate to resolve, the judgment be a situation that it would be appropriate to resolve, the judgment
as to how pressing this issue should be considered is a judgment for as to how pressing this issue should be considered is a judgment for
the reader, and eventually the nfsv4 working group to make. the reader, and eventually the nfsv4 working group to make.
We do explore possible ways in which such issues can be dealt with, We do explore possible ways in which such issues can be dealt with,
with minimal negative effects, given that the working group has with minimal negative effects, given that the working group has
skipping to change at page 5, line 6 skipping to change at page 5, line 11
whether the quotation is from: whether the quotation is from:
o The base definition of the NFSv4.0 protocol [RFC7530]. o The base definition of the NFSv4.0 protocol [RFC7530].
o The document updating the handling of migration in NFSv4.0 o The document updating the handling of migration in NFSv4.0
[RFC7931]. [RFC7931].
o The current definition of the NFSv4.1 protocol [RFC5661]. o The current definition of the NFSv4.1 protocol [RFC5661].
An additional possibility is that these terms may appear in a An additional possibility is that these terms may appear in a
proposed or possible text to serve as a replacement for a current proposed or possible text to serve as a replacement for some part of
protocol specification. Sometimes, a number of possible alternative a current protocol specification. Sometimes, a number of possible
texts may be listed and benefits and detriments of each examined in alternative texts may be listed and benefits and detriments of each
turn. examined in turn.
2.3. Terminology Used in this Document 2.3. Terminology Used in this Document
In this document the phrase "client ID" always refers to the 64-bit In this document the phrase "client ID" always refers to the 64-bit
shorthand identifier assigned by the server (a clientid4) and never shorthand identifier assigned by the server (a clientid4) and never
to the structure which the client uses to identify itself to the to the structure which the client uses to identify itself to the
server (called an nfs_client_id4 or client_owner in NFSv4.0 and server (called an nfs_client_id4 or client_owner in NFSv4.0 and
NFSv4.1 respectively). The opaque identifier within those structures NFSv4.1 respectively). The opaque identifier within those structures
is referred to as a client id string". is referred to as a client id string".
skipping to change at page 5, line 31 skipping to change at page 5, line 36
following terminology: following terminology:
o The phrase "connection type" denotes the use of an existing or o The phrase "connection type" denotes the use of an existing or
potential connection to support NFSv4 layered on top of the RPC potential connection to support NFSv4 layered on top of the RPC
stream transport as described in [RFC5531] or on top of RPC-over- stream transport as described in [RFC5531] or on top of RPC-over-
RDMA as described in [RFC8166]. Establishing a connection of a RDMA as described in [RFC8166]. Establishing a connection of a
particular type requires that the client and server support that particular type requires that the client and server support that
connection type given the particular client and server network connection type given the particular client and server network
addresses used. addresses used.
o Each connection is established between a client and a specfic o Each connection is established between a client and a specific
server endpoint. Two endpoints are considered distinct if they server endpoint. Two endpoints are considered distinct if they
differ in either network address or connection type. Multiple differ in either network address or connection type. Multiple
connections may be established to the same endpoint or to connections may be established to the same endpoint or to
different endpoints. different endpoints.
o The phrase "network endpoint specification" refers to the o The phrase "network endpoint specification" refers to the
combination of a network address and a connection type. combination of a network address and a connection type.
Regarding trunking of connections to server network endpoints, we use Regarding trunking of connections to server network endpoints, we use
the following terminology: the following terminology:
skipping to change at page 7, line 5 skipping to change at page 7, line 8
Each set of server-trunkable location elements defines the available Each set of server-trunkable location elements defines the available
access paths to a particular file system. When there are multiple access paths to a particular file system. When there are multiple
such file systems, each of these, which contains the same data, is a such file systems, each of these, which contains the same data, is a
replica of the others. Logically, such replication is symmetric, replica of the others. Logically, such replication is symmetric,
since the fs currently in use and an alternate fs are replicas of since the fs currently in use and an alternate fs are replicas of
each other. Often, in other documents, the term "replica" is not each other. Often, in other documents, the term "replica" is not
applied to the fs currently in use, despite the fact that the applied to the fs currently in use, despite the fact that the
replication relation is inherently symmetric. replication relation is inherently symmetric.
3. Issues that Apply to Multiple Versions and their Resolution 3. Issues that Apply to Multiple Minor Versions and their Resolution
Although there are a common set of issues that need to be addressed, Although there are a common set of issues that need to be addressed,
the diffences between NFSv4.0 and NFSv4.1 means that the detailed the differences between NFSv4.0 and NFSv4.1 means that the detailed
handling of these issues will be significantly different in each handling of these issues will be significantly different in each
prototol. protocol.
In order to accommodate this situation, this section will deal with In order to accommodate this situation, this section will deal with
the commonalities across protocol version while the specfics the commonalities across protocol minor versions while the specifics
appropriate to each protocol version are dealt with in Sections 4 and appropriate to each minor version are dealt with in Sections 4 and 5
5 respectively. respectively.
3.1. Issue Summary 3.1. Issue Summary
Many of these issues arise from a lack of clarity regarding the Many of these issues arise from a lack of clarity regarding the
meaning of and proper handling for location attributes that specify meaning of and proper handling for location attributes that specify
more than a single server address. Such situations can arise as a more than a single server address. Such situations can arise as a
result of multiple entries in the same attribute or because a single result of multiple entries in the same attribute or because a single
entry has a server name which, when processed by DNS, is mapped to entry has a server name which, when processed by DNS, is mapped to
multiple server addresses. multiple server addresses.
Another set of issues arises from the fact that many of the Another set of issues arises from the fact that many of the
facilities thst must deal with multiple network addresses assume facilities that must deal with multiple network addresses assume
there is only a single connection type shared by all of the there is only a single connection type shared by all of the
addresses. It is necessary to deal with a mixture of connection addresses. It is necessary to deal with a mixture of connection
types. types.
Both [RFC7530] and [RFC5661] indicate that multiple addresses may be Both [RFC7530] and [RFC5661] indicate that multiple addresses may be
present and that these addresses may be different paths to the same present and that these addresses may be different paths to the same
server as well as different copies of the same data. However, the server as well as different copies of the same data. However, the
following issues have, for both protocols, interfered with the following issues have, for both protocols, interfered with the
recognition of the existing location attributes as a way of providing recognition of the existing location attributes as a way of providing
a trunking discovery function: a trunking discovery function:
skipping to change at page 8, line 23 skipping to change at page 8, line 25
suggested in [RFC7931] but it is a suggestion only valid when the suggested in [RFC7931] but it is a suggestion only valid when the
client chooses to use the uniform client id string model. client chooses to use the uniform client id string model.
o For NFSv4.1, as described by [RFC5661], there is a standard method o For NFSv4.1, as described by [RFC5661], there is a standard method
of trunking detection, which can be relied upon. of trunking detection, which can be relied upon.
The issues that need to be addressed for both versions are: The issues that need to be addressed for both versions are:
o Provision of a trunking discovery facility to allow a client to o Provision of a trunking discovery facility to allow a client to
find out about other addresses that may be used to access the find out about other addresses that may be used to access the
current server. current server. Such a facility needs to allows the set of
addresses to be used to change as necessary.
o Discussion of how the appropriate connection type for a given o Discussion of how the appropriate connection type for a given
client-server connection is to be arrived at and of trunking client-server connection is to be arrived at and of trunking
issues between endpoints of multiple connection types using the issues between endpoints of multiple connection types using the
same network address. same network address.
o Better integration of migration with trunking changes, including o Better integration of migration with trunking changes, including
situations in which the set of endpoints usable to access the same situations in which the set of endpoints usable to access the same
server changes (without migration) and those in which there is a server changes (without migration) and those in which there is a
shift to a different server, but trunking of endpoints on either shift to a different server, but trunking of endpoints on either
the source or destination is involved the source or destination is involved
Note that although these issues need to be addressed for both Note that although these issues need to be addressed for both
protocols, the resolutiions need not be the same and the protocol protocols, the resolutions need not be the same and the protocol
facilities within each protocol may limit the completeness of the facilities within each protocol may limit the completeness of the
resolutions provided. resolutions provided.
3.2. Resolution of Multi-Version Issues 3.2. Resolution of Multi-Version Issues
Although the specifics of addressing these issues will be different Although the specifics of addressing these issues will be different
for different versions, there are some common aspects discussed in for different versions, there are some common aspects discussed in
the subsections below: the subsections below:
o The trunking discovery function is to be addressed in o The trunking discovery function is to be addressed in
substantially the same way in both versions, as explained in substantially the same way in both versions, as explained in
Section 3.2.1. The only version-related differences are the Section 3.2.1. The only version-related differences are the
inclusion of the fs_locations_info attribute in NFSv4.1 and the inclusion of the fs_locations_info attribute in NFSv4.1 and the
potential addition of further per-endpoint information within potential addition of further per-endpoint information within
extensions to be defined for use in later versions of NFSv4. extensions to be defined for use in later versions of NFSv4.
o Handling of changes to the set of addresses to be used also needs
to be addressed in substantially the same way in both versions.
See Section 3.2.2 for further discussion.
o The interaction of trunking and migration is discussed in general o The interaction of trunking and migration is discussed in general
terms in Section 3.2.2. However, the specifics of the NFSv4.1 terms in Section 3.2.3. However, the specifics of the NFSv4.1
client's response to NFS4ERR_MOVED are discussed in Sections client's response to NFS4ERR_MOVED are discussed in Sections
5.4.3, 5.4.4, and 5.4.5. 5.5.3, 5.5.4, and 5.5.5.
3.2.1. Providing Trunking Discovery 3.2.1. Providing Trunking Discovery
A client can discover a set network addresses to use to access a file A client can discover a set network addresses to use to access a file
system using an NFSv4 server in a number of ways: system using an NFSv4 server in a number of ways:
o If the client is accessing a server using its name, that name can o If the client is accessing a server using its name, that name can
be mapped to a set of IP addresses using DNS and if multiple be mapped to a set of IP addresses using DNS and if multiple
addresses are available, those addresses can generally be used addresses are available, those addresses can generally be used
together to access the server. together to access the server.
skipping to change at page 9, line 31 skipping to change at page 9, line 39
obtain the value of a location attribute (e.g. fs_locations). obtain the value of a location attribute (e.g. fs_locations).
Where an entry within that attribute specifies a server name, DNS Where an entry within that attribute specifies a server name, DNS
can be used to obtain one or more network addresses corresponding can be used to obtain one or more network addresses corresponding
to that name. In cases in which one of those is the address being to that name. In cases in which one of those is the address being
used, the others that corresponding to that name can also be used used, the others that corresponding to that name can also be used
to access the server. to access the server.
o A client can obtain the value of a location attribute (e.g. o A client can obtain the value of a location attribute (e.g.
fs_locations) and use location entries that specify network fs_locations) and use location entries that specify network
addresses. When there is a means of trunking detection available addresses. When there is a means of trunking detection available
(see below), all of addresses that are determined to correspond to (see below), all of the addresses that are determined to
the same server can be used to access the server. correspond to the same server can be used to access that server.
Note that the last two of these are usable in situations in which Note that the last two of these are usable in situations in which
NFS4ERR_MOVED was returned. Note that this does not necessarily mean NFS4ERR_MOVED was returned. Note that this does not necessarily mean
that migration has occurred since there may be a shift in the set of that migration has occurred since there may be a shift in the set of
network addresses to be used without changing to a different server. network addresses to be used without changing to a different server.
See Section 3.2.2 for further discussion. See Section 3.2.3 for further discussion.
Which of the above means of providing trunking information is Which of the above means of providing trunking information is
appropriate to use in a given environment will depend on security appropriate to use in a given environment will depend on security
considerations, the possible need for the server to direct different considerations, the possible need for the server to direct different
clients to different sets of addresses, and the availability of clients to different sets of addresses, and the availability of
trunking detection facilities on the clients. trunking detection facilities on the clients.
With regard to security, the possibility that requests to determine With regard to security, the possibility that requests to determine
the set of network addresses corresponding to a given server might be the set of network addresses corresponding to a given server might be
interfered with or have their responses corrupted needs to be taken interfered with or have their responses corrupted needs to be taken
skipping to change at page 10, line 42 skipping to change at page 11, line 5
o For NFSv4.1, the client can compare the server_owner returned in o For NFSv4.1, the client can compare the server_owner returned in
the response to EXCHANGE_ID to determine if two network addresses the response to EXCHANGE_ID to determine if two network addresses
correspond to the same server. correspond to the same server.
As a result, direct presentation of network addresses in location As a result, direct presentation of network addresses in location
entries may be problematic for NFSv4.0, since some clients might not entries may be problematic for NFSv4.0, since some clients might not
have the trunking detection facilities that allow them to take have the trunking detection facilities that allow them to take
advantage of this information. For further discussion of issues advantage of this information. For further discussion of issues
related to NFSv4.0, see Section 4.4. related to NFSv4.0, see Section 4.4.
3.2.2. Interaction of Trunking and Migration 3.2.2. Addressing Changes in Trunking Configuration
When the client is capable of finding out a set of network addresses
to use in accessing a server, it is always possible for that set to
change.
This sometimes requires that a network address previously used to
access a server becomes invalid for that purpose. This requires a
way of notifying the client and a way for the client to adapt to this
change by using a new set of network addresses to access the server.
This will involve recovery much like that for migration although the
same server and file system is used throughout.
3.2.3. Interaction of Trunking and Migration
When the set of network addresses designated by a location attribute When the set of network addresses designated by a location attribute
changes, NFS4ERR_MOVED may or may not result, and in some of the changes, NFS4ERR_MOVED may or may not result, and in some of the
cases in which it is returned migration will occur, while in others cases in which it is returned migration will occur, while in others
there a shift in the network addresses used to access a particular there a shift in the network addresses used to access a particular
file system with no migration. file system with no migration.
o When the list of networks addresses is a superset of that o When the list of networks addresses is a superset of that
previously in effect, there is no need for migration or any other previously in effect, there is no need for migration or any other
sort of client adjustment. Nevertheless, the client is free to sort of client adjustment. Nevertheless, the client is free to
use an additional address if it provides another path to the same use an additional address if it provides another path to the same
server. If, on the other hand, it does not do so, the client may server. If, on the other hand, it does not do so, the client may
treat it as it does a replica, to be used if the current server treat it as it does a separate replica, to be used if the current
addresses become unavailable. server addresses become unavailable.
o When the list of networks addresses is a subset of that previously o When the list of networks addresses is a subset of that previously
in effect, immediate action is not needed if the address was not in effect, immediate action is not needed if the address was not
being used. The client should avoid using it in the future, being used. The client should avoid using it in the future,
whether the address is for a replica or a potential additional whether the address is for another replica or a potential
path to the server being used. additional path to the server being used.
o When an address being removed is one of a number of paths to the o When an address being removed is one of a number of paths to the
current server, the client can cease to use it but it can continue current server, the client can cease to use it but it can continue
to use it until NFS4ERR_MOVED is received. This is not considered to use it until NFS4ERR_MOVED is received. This is not considered
a migration event, unless it is the last available path to the a migration event, unless it is the last available path to the
server that has become unusable. server that has become unusable.
When migration does occur, multiple addresses may be in use on the When migration does occur, multiple addresses may be in use on the
server previous to migration and multiple addresses may be available server previous to migration and multiple addresses may be available
for use on the destination server. for use on the destination server.
skipping to change at page 12, line 26 skipping to change at page 13, line 5
addresses usable with the current server or a migration target addresses usable with the current server or a migration target
before those associated with replicas. before those associated with replicas.
o A client can cease scanning for trunkable location entries once it o A client can cease scanning for trunkable location entries once it
encounters one whose fs_name differs from the current fs name. encounters one whose fs_name differs from the current fs name.
o A client can cease scanning for trunkable location entries once it o A client can cease scanning for trunkable location entries once it
encounters a location element whose address in not server- encounters a location element whose address in not server-
trunkable with the one it is using. trunkable with the one it is using.
3.2.3. Dealing with Multiple Connection Types 3.2.4. Dealing with Multiple Connection Types
Because of the use of RPC-over-RDMA [RFC8166] as an inderlying Because of the use of RPC-over-RDMA [RFC8166] as an underlying
transport for NFSV4, as described in [RFC8267], a client may have transport for NFSV4, as described in [RFC8267], a client may have
mutiple connection types to the same server network address. This multiple connection types to the same server network address. This
gives rise to a number of issues with regard to NFSv4 multi-server gives rise to a number of issues with regard to NFSv4 multi-server
namespace features. namespace features.
o In the case of migration or referral, the client is directed to o In the case of migration or referral, the client is directed to
one or more server network addresses and faces the problem of one or more server network addresses and faces the problem of
selecting the appropriate connection type. selecting the appropriate connection type.
o When trunking multiple connections, the client might be directed o When trunking multiple connections, the client might be directed
to use the same server network address with a different set of of to use the same server network address with a different set of
potential connection types leaving the client to choose the potential connection types leaving the client to choose the
connection type to be used when a set with multiple connection connection type to be used when a set with multiple connection
types is provided. types is provided.
Although the situation is similar for both protocol versions, Although the situation is similar for both protocol versions,
differences in the attributes supported may result in important differences in the attributes supported may result in important
differences in how connection types are selected. differences in how connection types are selected.
o In the case of NFSv4.0, the fs_locations attribute has no ability o In the case of NFSv4.0, the fs_locations attribute has no ability
to indicate valid connection types. Only the network address is to indicate valid connection types. Only the network address is
skipping to change at page 13, line 25 skipping to change at page 13, line 52
detecting whether RDMA support is present. When that flag is not detecting whether RDMA support is present. When that flag is not
present, step-up is not supported, but the client may use the present, step-up is not supported, but the client may use the
FSLI4TF_RDMA flag to determine if RDMA support is available, and FSLI4TF_RDMA flag to determine if RDMA support is available, and
establish a new connection to use to obtain RDMA support. establish a new connection to use to obtain RDMA support.
In addition to the selection of an appropriate connection type to use In addition to the selection of an appropriate connection type to use
when multiple connection types are available, the simultaneous when multiple connection types are available, the simultaneous
availability of multiple connection types raises issues related to availability of multiple connection types raises issues related to
trunking, in the same way as the availability of multiple network trunking, in the same way as the availability of multiple network
addresses connected to the same server. These issues, including the addresses connected to the same server. These issues, including the
relationship of such trunking to migration might be dealt with could relationship of such trunking to migration, might could potentially
potentially be dealt sdifferently within NFSv4.0 and NFSv4.1, be dealt differently within NFSv4.0 and NFSv4.1, although similar
although similar treatment is desirabble. The treatment of these treatment is desirable. The treatment of these issues is discussed
issues is discused in Sections 4.4.1 and 5.1.7 respectively. in Sections 4.4.1 and 5.2.5 respectively.
Note that the handling of trunking for NFSv4.0 and for an NFSv4.1 Note that the handling of trunking for NFSv4.0 and for an NFSv4.1
metadata server differs from that for an NFSv4.1 data server. In metadata server differs from that for an NFSv4.1 data server. In
that latter case, specification of trunking patterns including the that latter case, specification of trunking patterns including the
connection type of endpoints is under the control of the metadata connection type of endpoints is under the control of the metadata
server and the client simply uses the information presented by the server and the client simply uses the information presented by the
metadata server to guide selection of the endpoints to be accessed. metadata server to guide selection of the endpoints to be accessed.
One potential difference between the versions that needs to be One potential difference between the versions that needs to be
resolved concerns the issue of the trunking of multiple connections resolved concerns the issue of the trunking of multiple connections
directed to endpoints that share a network address while differing as directed to endpoints that share a network address while differing as
to connection type. While NFSv4.1 is specified in [RFC5661] as to connection type. While NFSv4.1 is specified in [RFC5661] as
requring that such connections be trunkable, neither [RFC7530] nor requiring that such connections be trunkable, neither [RFC7530] nor
[RFC7931] contains a corresponding statement. [RFC7931] contains a corresponding statement.
4. NFSv4.0 Issues 4. NFSv4.0 Issues
4.1. Core NFSv4.0 Migration Issues 4.1. Core NFSv4.0 Migration Issues
Many of the problems seen with Transparent State Migration derived Many of the problems seen with Transparent State Migration derived
from the inability of NFSv4.0servers to determine whether two client from the inability of NFSv4.0servers to determine whether two client
IDs, issued on different servers, corresponded to the same client. IDs, issued on different servers, corresponded to the same client.
This difficulty derived in turn from the common practice, recommended This difficulty derived in turn from the common practice, recommended
skipping to change at page 15, line 37 skipping to change at page 16, line 18
were addressed by the publication of [RFC7931], the remaining issues were addressed by the publication of [RFC7931], the remaining issues
derive from those mentioned in Section 3.1. These include: derive from those mentioned in Section 3.1. These include:
o Introducing facilities for trunking discovery. o Introducing facilities for trunking discovery.
o Clarifying the handling of multiple connection types, including o Clarifying the handling of multiple connection types, including
issues related to the trunking of multiple connections of issues related to the trunking of multiple connections of
different types to the same network address. different types to the same network address.
o Clarifying the relationship between migration and trunking, o Clarifying the relationship between migration and trunking,
including trunking among multple server endpoints sharing a server including trunking among multiple server endpoints sharing a
network address. server network address.
4.4. Resolution of Additional NFSv4.0 Issues 4.4. Resolution of Additional NFSv4.0 Issues
One possible approach to addressing these issues would entail One possible approach to addressing these issues would entail
publication of an additional standards-track document updating publication of an additional standards-track document updating
[RFC7530]. [RFC7530].
Fortunately, it appears that all of the material to be updated Fortunately, it appears that all of the material to be updated
appears in Section 8 of that document, whether it concerns the appears in Section 8 of that document, whether it concerns the
provision of trunking discovery or the interaction of trunking and provision of trunking discovery or the interaction of trunking and
skipping to change at page 17, line 19 skipping to change at page 17, line 44
o A replacement subsection dealing with replication and trunking. o A replacement subsection dealing with replication and trunking.
o A replacement subsection dealing with migration. o A replacement subsection dealing with migration.
o A new subsection dealing with the interaction of trunking, o A new subsection dealing with the interaction of trunking,
replication, and migration. replication, and migration.
4.4.1. Resolution of NFSv4.0 Issues with Multiple Connection Types 4.4.1. Resolution of NFSv4.0 Issues with Multiple Connection Types
The existence of multiple connection types raises issues regarding The existence of multiple connection types raises issues regarding
how the connection type to be used is determied by the client. Such how the connection type to be used is determined by the client. Such
issues need to be addressed when a new server is accessed and also issues need to be addressed when a new server is accessed and also
when NFS4ERR_MOVED is returned and a server endpoint is to be when NFS4ERR_MOVED is returned and a server endpoint is to be
selected to access the current file system. selected to access the current file system.
The absence of explicit support for multiple connection types within The absence of explicit support for multiple connection types within
NFSv4.0 means that the client has a great deal of freedom in making NFSv4.0 means that the client has a great deal of freedom in making
this determination, although some implementation guidance could be this determination, although some implementation guidance could be
provided. A client could attempt to establish a connection of each provided. A client could attempt to establish a connection of each
connection type and the connection type (or types) that it chooses. connection type and the connection type (or types) that it chooses.
To make this an efficient process, servers which do not provide To make this an efficient process, servers which do not provide
skipping to change at page 17, line 47 skipping to change at page 18, line 24
cases of migration and referral, as well as for initial mount. cases of migration and referral, as well as for initial mount.
Clients might well treat these situations differently, for example by Clients might well treat these situations differently, for example by
using the type of the current connection as the initial type to try using the type of the current connection as the initial type to try
in the migration case, while not doing in other cases. in the migration case, while not doing in other cases.
Situations in which NFS4ERR_MOVED is returned without requiring any Situations in which NFS4ERR_MOVED is returned without requiring any
shift in target network address require special attention, in order shift in target network address require special attention, in order
to allow a shift in the network endpoint to be used to be indicated to allow a shift in the network endpoint to be used to be indicated
even if there is no corresponding shift in network address. In the even if there is no corresponding shift in network address. In the
absence of multiple connection types, receiving NFS4ERR_MOVED when absence of multiple connection types, receiving NFS4ERR_MOVED when
acessing one file system serves as an indication that that address is accessing one file system serves as an indication that that address
not to be used to access that file system subsequently, making it is not to be used to access that file system subsequently, making it
necessary to use other network addresses to access the file system, necessary to use other network addresses to access the file system,
after migration or a shift in trunking patterns without migration. after migration or a shift in trunking patterns without migration.
Since NFSv4.0 does not provide any way for the server to specify the Since NFSv4.0 does not provide any way for the server to specify the
use of particular connection types, it might seem that there is no use of particular connection types, it might seem that there is no
way for the server to direct such a shift. However, when way for the server to direct such a shift. However, when
NFS4ERR_MOVED is returned and the network address on which it was NFS4ERR_MOVED is returned and the network address on which it was
returned is still present in the location entries returned, a client returned is still present in the location entries returned, a client
may reasonably conclude that: may reasonably conclude that:
o The endpoint from whic NFS4ERR_MOVED was returned is not to be o The endpoint from which NFS4ERR_MOVED was returned is not to be
used to acess the file system in question. used to access the file system in question.
o Other endpoints using the same network address but different o Other endpoints using the same network address but different
connection types could be used to access the filesystem. connection types could be used to access the filesystem.
This gives the client a set of server endpoints to test for access to This gives the client a set of server endpoints to test for access to
the filsystem. In cases in which there is already a connection the filesystem. In cases in which there is already a connection
established to that endpoint, file system access can be tested using established to that endpoint, file system access can be tested using
a PUTFH within the target file system followed by a GETFH, which will a PUTFH within the target file system followed by a GETFH, which will
either succeed or return NFS4ERR_MOVED depending on whether the either succeed or return NFS4ERR_MOVED depending on whether the
endpoint used can validly access the file system. In other cases a endpoint used can validly access the file system. In other cases a
connection will need to be established before such a test can be connection will need to be established before such a test can be
performed. performed.
5. Issues for NFSv4.1 and Beyond 5. Issues for NFSv4.1 and Beyond
5.1. Issues to Address for NFSv4.1 5.1. Trunking-focused Issues to Address for NFSv4.1
While the addition of trunking discovery will be addressed in the
same way for both protocols, there are a number of cases in which
there are issues where the specifics of v4.1 require special
treatment:
o Because there is information in the fs_locations_info attribute
that goes beyond the location entries, it needs to be clear how
such information is to be handled, particularly when it applies to
multiple access paths which might or might not be paths to the
same replica. See Section 5.1.1 for a discussion of related
issues.
o Since NFSv4.1 provides a way to determine whether two addresses
are connected to the same server, the possibility that the
information on which that determination will change needs to be
addressed, as is discussed in Section 5.1.2.
5.1.1. Handling of Additional Information in fs_locations_info
The more extensive structure of the fs_locations_info attribute, as
compared with fs_locations, means that a number of areas may need
clarification, when fs_locations_info is used in connection with
trunking discovery:
o The function of fields within the attribute, outside the location
entries (e.g. fli_valid_for) may need to be clarified, as it
applies to trunking discovery.
o Since much of the information in the fs_locations4_server
structure applies to a particular replica, rather than a specific
access path, it needs to be clear where such information is
replica-specific or access-path-specific and how any
inconsistencies are to be dealt with.
5.1.2. Addressing Server_owner Changes in NFSv4.1
This issue is addressed in [RFC5661], although it does not provide a
clear description of the needed handling.
Section 2.10.5 of [RFC5661] states the following.
The client should be prepared for the possibility that
eir_server_owner values may be different on subsequent EXCHANGE_ID
requests made to the same network address, as a result of various
sorts of reconfiguration events. When this happens and the
changes result in the invalidation of previously valid forms of
trunking, the client should cease to use those forms, either by
dropping connections or by adding sessions. For a discussion of
lock reclaim as it relates to such reconfiguration events, see
Section 8.4.2.1.
While this paragraph is literally true in that such reconfiguration
events can happen and clients have to deal with them, it is confusing
in that it can be read as suggesting that clients have to deal with
them without disruption, which in general is impossible.
A clearer alternative would be:
It is always possible that, as a result of various sorts of
reconfiguration events, eir_server_scope and eir_server_owner
values may be different on subsequent EXCHANGE_ID requests made to
the same network address.
In most cases such reconfiguration events will be disruptive and
indicate that an IP address formerly connected to one server is
now connected to an entirely different one.
Some guidelines on client handling of such situations follow:
o When eir_server_scope changes, the client has no assurance that
any id's it obtained previously (e.g. file handles) can be
validly used on the new server, and, even if the new server
accepts them, there is no assurance that this is not due to
accident. Thus it is best to treat all such state as lost/
stale although a client may assume that the probability of
inadvertent acceptance is low and treat this situation as
within the next case.
o When eir_server_scope remains the same and
eir_server_owner.so_major_id changes, the client can use
filehandles it has and attempt reclaims. It may find that
these are now stale but if NFS4ERR_STALE is not received, he
can proceed to reclaim his opens.
o When eir_server_scope and eir_server_owner.so_major_id remain
the same, the client has to use the now-current values of
eir_server-owner.so_minor_id in deciding on appropriate forms
of trunking.
5.2. Migration-related Issues to Address for NFSv4.1
Because NFSv4.1 embraces the uniform client-string approach, as Because NFSv4.1 embraces the uniform client-string approach, as
advised by section 2.4 of [RFC5661], addressing migration issues is advised by section 2.4 of [RFC5661], addressing migration issues is
simpler, in that a shift in client id string models is not required. simpler, in that a shift in client id string models is not required.
Instead, NFSv4 returns information in the EXCHANGE_ID response to Instead, NFSv4 returns information in the EXCHANGE_ID response to
enable trunking relationships to be determined by the client. enable trunking relationships to be determined by the client.
Despite this simplification, there are substantial issues that need Despite this simplification, there are substantial issues that need
to be dealt with: to be dealt with:
skipping to change at page 19, line 5 skipping to change at page 21, line 29
made to make it clear that state needs to be appropriately merged made to make it clear that state needs to be appropriately merged
as part of migration, to avoid multiple client IDs between a as part of migration, to avoid multiple client IDs between a
client-server pair. client-server pair.
o The current discussion (in [RFC5661]), of the possibility of o The current discussion (in [RFC5661]), of the possibility of
server_owner changes is incomplete and confusing. server_owner changes is incomplete and confusing.
o As with NFSV4.0, the interaction of trunking with migration and o As with NFSV4.0, the interaction of trunking with migration and
other aspects of multi-server namespace needs to be clarified. other aspects of multi-server namespace needs to be clarified.
Addressing migration in NFSv41 will also require adaptation of the Addressing migration in NFSv4.1 will also require adaptation of the
approaches used in [RFC7931] to the NFSv4.1 environment including: approaches used in [RFC7931] to the NFSv4.1 environment including:
o The use of EXCHANGE_ID needs to be accommodated including issues o The use of EXCHANGE_ID needs to be accommodated including issues
associated with the expected confirmation status of client IDs associated with the expected confirmation status of client IDs
transferred by Transparent State Migration. transferred by Transparent State Migration.
o The use of sessions needs to be addressed including discussion of o The use of sessions needs to be addressed including discussion of
the proper use of the status bits returned by the SEQUENCE the proper use of the status bits returned by the SEQUENCE
operation. operation.
skipping to change at page 19, line 29 skipping to change at page 22, line 5
o There needs to be some clarification of how migration, and o There needs to be some clarification of how migration, and
particularly Transparent State Migration, should interact with particularly Transparent State Migration, should interact with
pNFS layouts. pNFS layouts.
o There are a number of issues related to the migration of sessions o There are a number of issues related to the migration of sessions
that need to be addressed. that need to be addressed.
Discussion of how to resolve these issues will appear in the sections Discussion of how to resolve these issues will appear in the sections
below. below.
5.1.1. Addressing State Merger in NFSv4.1 5.2.1. Addressing State Merger in NFSv4.1
The existing treatment of state transfer in [RFC5661], has similar The existing treatment of state transfer in [RFC5661], has similar
problems to that in [RFC7530] in that it assumes that the state for problems to that in [RFC7530] in that it assumes that the state for
multiple filesystems formerly on different servers will not be merged multiple filesystems formerly on different servers will not be merged
so that it appears under a single common client ID. We've already so that it appears under a single common client ID. We've already
seen the reasons that this is a problem with regard to NFSv4.0. seen the reasons that this is a problem with regard to NFSv4.0.
Although we don't have the problems stemming from the non-uniform Although we don't have the problems stemming from the non-uniform
client-string approach, there are a number of complexities in the client-string approach, there are a number of complexities in the
existing treatment of state management in the section entitled "Lock existing treatment of state management in the section entitled "Lock
skipping to change at page 20, line 11 skipping to change at page 22, line 35
o In the case of matching server scopes, the text calls for an o In the case of matching server scopes, the text calls for an
unrealistic degree of transparency, suggesting that the source and unrealistic degree of transparency, suggesting that the source and
destination servers need to cooperate in stateid and client ID destination servers need to cooperate in stateid and client ID
assignment. assignment.
o In the case of non-matching server scopes, the text does not o In the case of non-matching server scopes, the text does not
mention the possibility of the transparent migration of state at mention the possibility of the transparent migration of state at
all, resulting in a functional regression from NFSV4.0 all, resulting in a functional regression from NFSV4.0
5.1.2. Addressing pNFS Relationship with Migration 5.2.2. Addressing pNFS Relationship with Migration
This is made difficult because, within the pNFS framework, migration This is made difficult because, within the pNFS framework, migration
might mean any of several things: might mean any of several things:
o Transfer of the MDS, leaving DS's as they are. o Transfer of the MDS, leaving DS's as they are.
This would be minimally disruptive to those using layouts but This would be minimally disruptive to those using layouts but
would require the pNFS control protocol being used to support the would require the pNFS control protocol being used to support the
DS being directed to a new MDS. DS being directed to a new MDS.
skipping to change at page 20, line 37 skipping to change at page 23, line 14
o Transfer of the filesystem to a new filesystem with both MDS and o Transfer of the filesystem to a new filesystem with both MDS and
DS's moving. DS's moving.
In such a transfer, an entirely different set of DS's will be at In such a transfer, an entirely different set of DS's will be at
the target location. There may even be no pNFS support on the the target location. There may even be no pNFS support on the
destination filesystem at all. destination filesystem at all.
Migration needs to support both the first and last of these models. Migration needs to support both the first and last of these models.
5.1.3. Addressing Server_owner Changes in NFSv4.1 5.2.3. Addressing Confirmation Status of Migrated Client IDs in NFSv4.1
Section 2.10.5 of [RFC5661] states the following.
The client should be prepared for the possibility that
eir_server_owner values may be different on subsequent EXCHANGE_ID
requests made to the same network address, as a result of various
sorts of reconfiguration events. When this happens and the
changes result in the invalidation of previously valid forms of
trunking, the client should cease to use those forms, either by
dropping connections or by adding sessions. For a discussion of
lock reclaim as it relates to such reconfiguration events, see
Section 8.4.2.1.
While this paragraph is literally true in that such reconfiguration
events can happen and clients have to deal with them, it is confusing
in that it can be read as suggesting that clients have to deal with
them without disruption, which in general is impossible.
A clearer alternative would be:
It is always possible that, as a result of various sorts of
reconfiguration events, eir_server_scope and eir_server_owner
values may be different on subsequent EXCHANGE_ID requests made to
the same network address.
In most cases such reconfiguration events will be disruptive and
indicate that an IP address formerly connected to one server is
now connected to an entirely different one.
Some guidelines on client handling of such situations follow:
o When eir_server_scope changes, the client has no assurance that
any id's it obtained previously (e.g. file handles) can be
validly used on the new server, and, even if the new server
accepts them, there is no assurance that this is not due to
accident. Thus it is best to treat all such state as lost/
stale although a client may assume that the probability of
inadvertent acceptance is low and treat this situation as
within the next case.
o When eir_server_scope remains the same and
eir_server_owner.so_major_id changes, the client can use
filehandles it has and attempt reclaims. It may find that
these are now stale but if NFS4ERR_STALE is not received, he
can proceed to reclaim his opens.
o When eir_server_scope and eir_server_owner.so_major_id remain
the same, the client has to use the now-current values of
eir_server-owner.so_minor_id in deciding on appropriate forms
of trunking.
5.1.4. Addressing Confirmation Status of Migrated Client IDs in NFSv4.1
When a client ID is transferred between systems as a part of When a client ID is transferred between systems as a part of
migration, it has never been clear whether it should be considered migration, it has never been clear whether it should be considered
confirmed or unconfirmed on the target server. In the case in which confirmed or unconfirmed on the target server. In the case in which
an associated session is transferred together with the client ID, it an associated session is transferred together with the client ID, it
is clear that the transferred client ID needs to be considered is clear that the transferred client ID needs to be considered
confirmed, as the existence of an associated session is incompatible confirmed, as the existence of an associated session is incompatible
with an unconfirmed client ID. with an unconfirmed client ID.
The case in which a client ID is transferred without an associated The case in which a client ID is transferred without an associated
skipping to change at page 22, line 34 skipping to change at page 24, line 5
source server and the transfer is not considered to have affected source server and the transfer is not considered to have affected
that. Given the current description of EXCHANGE_ID in [RFC5661], that. Given the current description of EXCHANGE_ID in [RFC5661],
some modification in the treatment of client id confirmation is some modification in the treatment of client id confirmation is
called for. In particular, provision would have to be made to called for. In particular, provision would have to be made to
enable the client id slot sequence id to be used by the client to enable the client id slot sequence id to be used by the client to
be determined. be determined.
Although the first approach makes it simpler for the client to Although the first approach makes it simpler for the client to
determine whether there is an associated session transferred at the determine whether there is an associated session transferred at the
same time, it makes it more difficult to determine whether same time, it makes it more difficult to determine whether
Transparent State Migration has occurred. Section 5.1.6. Transparent State Migration has occurred. Section 5.2.4.
In any case, adjustments will be required to deal with the fact that In any case, adjustments will be required to deal with the fact that
[RFC5661] currently assumes that a client id can only be confirmed by [RFC5661] currently assumes that a client id can only be confirmed by
issuing a CREATE_SESSION. In order to properly deal with the status issuing a CREATE_SESSION. In order to properly deal with the status
of migrated client ids, we have to distinguish among: of migrated client ids, we have to distinguish among:
o The confirmation status as reported by EXCHANGE_ID. o The confirmation status as reported by EXCHANGE_ID.
o Whether the client id is considered confirmed as that term is used o Whether the client id is considered confirmed as that term is used
in the many other cases in which the confirmation status of a in the many other cases in which the confirmation status of a
client ID affects how requests are handled. client ID affects how requests are handled.
o How the client is to determine the initial sequence id to be used o How the client is to determine the initial sequence id to be used
when doing operations such as CREATE_SESSION. when doing operations such as CREATE_SESSION.
In [RFC5661] as it currently stands all of these are tied together In [RFC5661] as it currently stands all of these are tied together
and it is not obvious how migrated client IDs could be accommodated and it is not obvious how migrated client IDs could be accommodated
in this structure, and what changes are necessary to make this in this structure, and what changes are necessary to make this
possible. For more discussion of this issue, see Section 5.2.1. possible. For more discussion of this issue, see Section 5.3.1.
5.1.5. Addressing Changes in Trunking Configuration
When the client us capable of finding out a set of network addresses
to use in accessing a server, it is always possible for that set to
change.
This sometimes requires that a network address previously used to
access a server becomes invalid for that purpose. This requires a
way of notifying the client and a way for the client to adapt to this
change by using a new set of network addresses to access the server.
his will involve recovery much like hat for migration although the
same server and file system is used throughout.
5.1.6. Addressing Session Migration in NFSv4.1 5.2.4. Addressing Session Migration in NFSv4.1
Some issues that need to be addressed regard the migration of Some issues that need to be addressed regard the migration of
sessions, in addition to client IDs and stateids sessions, in addition to client IDs and stateids
o It needs to be made clearer how the client can deal with the o It needs to be made clearer how the client can deal with the
possibility that sessions might or might not be transferred as possibility that sessions might or might not be transferred as
part of Transparent State Migration. part of Transparent State Migration.
o Rules need to be clarified regarding possible transfer of sessions o Rules need to be clarified regarding possible transfer of sessions
when either the source session is being used to access other file when either the source session is being used to access other file
systems on source server or there is already a session connecting systems on source server or there is already a session connecting
the client to the destination server. the client to the destination server.
o There needs to be more detail regarding how the protocol avoids o There needs to be more detail regarding how the protocol avoids
situations in which the same session is subject to concurrent situations in which the same session is subject to concurrent
changes on two different servers at the same time. changes on two different servers at the same time.
5.1.7. Dealing with Multiple Connection Types in NFSv4.1 5.2.5. Dealing with Multiple Connection Types in NFSv4.1
The existence of multiple connection types raises issues regarding The existence of multiple connection types raises issues regarding
how the connection type to be used is determied by the client. Such how the connection type to be used is determined by the client. Such
issues need to be addressed when a new server is accessed and also issues need to be addressed when a new server is accessed and also
when NFS4ERR_MOVED is returned and a server endpoint is to be when NFS4ERR_MOVED is returned and a server endpoint is to be
selected to access the current file system. selected to access the current file system.
The limited support for multiple connection types within NFSv4.1 The limited support for multiple connection types within NFSv4.1
means that a client can make this determination by first establishing means that a client can make this determination by first establishing
a non-RDMA connection and then using the the FSLI4TF_RDMA flag in the a non-RDMA connection and then using the FSLI4TF_RDMA flag in the
fs_locations_info attribute for the root file system to determine if fs_locations_info attribute for the root file system to determine if
an RDMA conection should be established. Such a connection can then, an RDMA connection should be established. Such a connection can
at the client's option, replace or remain truked with the original then, at the client's option, replace or remain trunked with the
connection. As an alternative, where support is provided, the xxxx original connection.
flag returned by EXCHANGE_ID can be used to guide a trnsfer of the
existing connection to RDMA mode.
The approach mentioned above should, in general, be usable in the The approach mentioned above should, in general, be usable in the
cases of migration and referral, as well as for initial mount. cases of migration and referral, as well as for initial mount.
Situations in which NFS4ERR_MOVED is returned without requiring any Situations in which NFS4ERR_MOVED is returned without requiring any
shift in target network address require special attention, in order shift in target network address require special attention, in order
to allow a shift in the network endpoint to be used to be indicated to allow a shift in the network endpoint to be used to be indicated
even if there is no corresponding shift in network address. In the even if there is no corresponding shift in network address. In the
absence of multiple connection types, receiving NFS4ERR_MOVED when absence of multiple connection types, receiving NFS4ERR_MOVED when
acessing one file system serves as an indication that that address is accessing one file system serves as an indication that that address
not to be used to access that file system subsequently, making it is not to be used to access that file system subsequently, making it
necessary to use other network addresses to access the file system, necessary to use other network addresses to access the file system,
after migration or a shift in trunking patterns without migration. after migration or a shift in trunking patterns without migration.
Since NFSv4.1 only limited facilities for the server to specify the Since NFSv4.1 only limited facilities for the server to specify the
use of particular connection types, there are difficulties in use of particular connection types, there are difficulties in
directing such a shift. When NFS4ERR_MOVED is returned and the directing such a shift. When NFS4ERR_MOVED is returned and the
network address on which it was returned is still present in the network address on which it was returned is still present in the
location entries returned, a client may reasonably conclude that: location entries returned, a client may reasonably conclude that:
o The usability of the aassociated RDMA endpoint can be determined o The usability of the associated RDMA endpoint can be determined
based on the status of the the FSLI4TF_RDMA in the based on the status of the FSLI4TF_RDMA in the fs_locations_info
fs_locations_info attribute for the file system being accessed. attribute for the file system being accessed.
o The endpoint returning NFS4ERR_MOVED is not to be used to acess o The endpoint returning NFS4ERR_MOVED is not to be used to access
the file system in question. the file system in question.
o Other endpoints using the same network address but different o Other endpoints using the same network address but different
connection types could be used to access the filesystem. connection types could be used to access the filesystem.
This generally allows client to determine set of server endpoints to This generally allows client to determine set of server endpoints to
be used to access the filsystem. In cases in which there is some be used to access the filesystem. In cases in which there is some
ambiguity file system access can be tested by establishing a ambiguity file system access can be tested by establishing a
connection if not already present and then using a PUTFH within the connection if not already present and then using a PUTFH within the
target file system followed by a GETFH, which will either succeed or target file system followed by a GETFH, which will either succeed or
return NFS4ERR_MOVED depending on whether the endpoint used can return NFS4ERR_MOVED depending on whether the endpoint used can
validly access the file system. validly access the file system.
5.2. Possible Resolutions for NFSv4.1 Protocol Issues 5.3. Possible Resolutions for NFSv4.1 Protocol Issues
The subsections below explore some ways of dealing with clarifying The subsections below explore some ways of dealing with clarifying
the protocol to address issues discussed in Section 5.1 the protocol to address issues discussed in Section 5.2
5.2.1. Client ID Confirmation Issues 5.3.1. Client ID Confirmation Issues
As mentioned previously [RFC5661], makes no provision for client IDs As mentioned previously [RFC5661], makes no provision for client IDs
that are confirmed other than through the use of CREATE_SESSION. For that are confirmed other than through the use of CREATE_SESSION. For
example Section 18.35 of [RFC5661] states: example Section 18.35 of [RFC5661] states:
The client uses the EXCHANGE_ID operation to register a particular The client uses the EXCHANGE_ID operation to register a particular
client owner with the server. The client ID returned from this client owner with the server. The client ID returned from this
operation will be necessary for requests that create state on the operation will be necessary for requests that create state on the
server and will serve as a parent object to sessions created by server and will serve as a parent object to sessions created by
the client. In order to confirm the client ID it must first be the client. In order to confirm the client ID it must first be
skipping to change at page 26, line 20 skipping to change at page 27, line 25
CREATE_SESSION is not needed to confirm it. Of course, subsequent CREATE_SESSION is not needed to confirm it. Of course, subsequent
CREATE_SESSION operations may be needed for other reasons. CREATE_SESSION operations may be needed for other reasons.
The value eir_seqenceid is used to establish an initial sequence The value eir_seqenceid is used to establish an initial sequence
value associate with the client ID returned. In cases in which a value associate with the client ID returned. In cases in which a
CREATE_SESSION has already been done, there is no need for this CREATE_SESSION has already been done, there is no need for this
value, since sequencing of such request has already been value, since sequencing of such request has already been
established and the client has no need for this value and will established and the client has no need for this value and will
ignore it ignore it
5.2.2. Dealing with Multiple Location Entries 5.3.2. Dealing with Multiple Location Entries
The possibility that more than one server address may be present in The possibility that more than one server address may be present in
location attributes requires further clarification. This is location attributes requires further clarification. This is
particularly the case, given the potential role of trunking for particularly the case, given the potential role of trunking for
NFSv4.1, whose connection to migration needs to be clarified. NFSv4.1, whose connection to migration needs to be clarified.
The description of the location attributes in [RFC5661], while it The description of the location attributes in [RFC5661], while it
indicates that multiple address entries in these attributes may be indicates that multiple address entries in these attributes may be
used to indicate alternate paths to the file system, does so mainly used to indicate alternate paths to the file system, does so mainly
in the context of replication and does so without mentioning in the context of replication and does so without mentioning
skipping to change at page 28, line 10 skipping to change at page 29, line 16
in the case in which the server seeks to direct traffic with in the case in which the server seeks to direct traffic with
regard to particular file systems to choose addresses, in the regard to particular file systems to choose addresses, in the
interest of load balancing, to adjust to hardware availability interest of load balancing, to adjust to hardware availability
constraints, or for other reasons. constraints, or for other reasons.
o In other cases, migration has occurred and the client can o In other cases, migration has occurred and the client can
determine whether Transparent State Migration occurred and whether determine whether Transparent State Migration occurred and whether
any locking state was lost during the transfer. any locking state was lost during the transfer.
Whether migration has occurred or not, the client can use the Whether migration has occurred or not, the client can use the
procedure described in Section 5.4.3 to recover access to existing procedure described in Section 5.5.3 to recover access to existing
locking state and, in some cases, sessions. locking state and, in some cases, sessions.
One should note the following differences between migration with One should note the following differences between migration with
Transparent State Migration and the similar cases in which there is a Transparent State Migration and the similar cases in which there is a
continuity of locking state with no change in the server. continuity of locking state with no change in the server.
o When locks are lost (as indicated when using them or via the o When locks are lost (as indicated when using them or via the
SEQ4_STATUS flags) and migration has not been done, they are not SEQ4_STATUS flags) and migration has not been done, they are not
to be reclaimed, except when SEQ4_STATUS_RESTART_RECLAIM_NEEDED is to be reclaimed, except when SEQ4_STATUS_RESTART_RECLAIM_NEEDED is
set. Instead such losses are treated as lock revocations and set. Instead such losses are treated as lock revocations and
acknowledged using FREE_STATEID. acknowledged using FREE_STATEID.
o When migration has not been done, there is no need for a o When migration has not been done, there is no need for a
RECLAIM_COMPLETE (with rca_one_fs set to true). RECLAIM_COMPLETE (with rca_one_fs set to TRUE).
5.2.3. Migration and pNFS 5.3.3. Migration and pNFS
When pNFS is involved, the protocol is capable of supporting: When pNFS is involved, the protocol is capable of supporting:
o Migration of the MDS, leaving DS's in place. o Migration of the MDS, leaving DS's in place.
o Migration of the file system as a whole, including the MDS and o Migration of the file system as a whole, including the MDS and
associated DS's. associated DS's.
o Replacement of one DS by another. o Replacement of one DS by another.
skipping to change at page 29, line 24 skipping to change at page 30, line 30
Replacement of one DS by another is not addressed by migration as Replacement of one DS by another is not addressed by migration as
such but can be effected by an MDS recalling layouts for the DS to be such but can be effected by an MDS recalling layouts for the DS to be
replaced and issuing new ones to be served by the successor DS. replaced and issuing new ones to be served by the successor DS.
Migration may transfer a file system from a server which does not Migration may transfer a file system from a server which does not
support pNFS to one which does. In order to properly adapt to this support pNFS to one which does. In order to properly adapt to this
situation, clients which support pNFS, but function adequately in its situation, clients which support pNFS, but function adequately in its
absence, should check for pNFS support when a file system is migrated absence, should check for pNFS support when a file system is migrated
and be prepared to use pNFS when support is available. and be prepared to use pNFS when support is available.
5.3. Defining Server Responsibilities for NFSv4.1 5.4. Defining Server Responsibilities for NFSv4.1 Transitions
The subsections below discuss the responsibilities of source and The subsections below discuss server responsibilities in providing
for the propagation of locking state when a file system is migrated.
Sections 5.4.1 and 5.4.2 discuss the responsibilities of source and
destination servers in effecting the necessary transfer of destination servers in effecting the necessary transfer of
information to support Transparent State Migration. information to support Transparent State Migration.
5.3.1. Server Responsibilities in Effecting Transparent State Migration Section 5.4.3 discusses issues relating to the handling of state
recovery using client-directed reclaim of existing locks, used when
Transparent State Migration is not available
5.4.1. Server Responsibilities in Effecting Transparent State Migration
The basic responsibility of the source server in effecting The basic responsibility of the source server in effecting
Transparent State Migration is to make available to the destination Transparent State Migration is to make available to the destination
server a description of each piece of locking state associated with server a description of each piece of locking state associated with
the file system being migrated. In addition to client id string and the file system being migrated. In addition to client id string and
verifier, the source server needs to provide, for each stateid: verifier, the source server needs to provide, for each stateid:
o The stateid including the current sequence value. o The stateid including the current sequence value.
o The associated client ID. o The associated client ID.
skipping to change at page 30, line 23 skipping to change at page 31, line 36
that locks revoked soon before or soon after migration are not that locks revoked soon before or soon after migration are not
inadvertently allowed to be reclaimed in situations in which the inadvertently allowed to be reclaimed in situations in which the
continuity of lock possession cannot be assured. continuity of lock possession cannot be assured.
o For locks lost on the source but whose loss has not yet been o For locks lost on the source but whose loss has not yet been
acknowledged by the client (by using FREE_STATEID), the acknowledged by the client (by using FREE_STATEID), the
destination must be aware of this loss so that it can deny a destination must be aware of this loss so that it can deny a
request to reclaim them. request to reclaim them.
o For locks lost on the destination after the state transfer but o For locks lost on the destination after the state transfer but
before the client's RECLAIM_COMPLTE is done, the destination before the client's RECLAIM_COMPLETE is done, the destination
server should note these and not allow them to be reclaimed. server should note these and not allow them to be reclaimed.
An additional responsibility of the cooperating servers concerns An additional responsibility of the cooperating servers concerns
situations in which a stateid cannot be transferred transparently situations in which a stateid cannot be transferred transparently
because it conflicts with an existing stateid held by the client and because it conflicts with an existing stateid held by the client and
associated with a different file system. In this case there are two associated with a different file system. In this case there are two
valid choices: valid choices:
o Treat the transfer, as in NFSv4.0, as one without Transparent o Treat the transfer, as in NFSv4.0, as one without Transparent
State Migration. In this case, conflicting locks cannot be State Migration. In this case, conflicting locks cannot be
granted until the client does a RECLAIM_COMPLETE, after reclaiming granted until the client does a RECLAIM_COMPLETE (with rca_one_fs
the locks it had, with the exception of reclaims denied because set to TRUE), after reclaiming the locks it had, with the
they were attempts to reclaim locks that had been lost. exception of reclaims denied because they were attempts to reclaim
locks that had been lost.
o Implement Transparent State Migration, except for the lock with o Implement Transparent State Migration, except for the lock with
the conflicting stateid. In this case, the client will be aware the conflicting stateid. In this case, the client will be aware
of a lost lock (through the SEQ4_STATUS flags) and be allowed to of a lost lock (through the SEQ4_STATUS flags) and be allowed to
reclaim it. reclaim it, after which it does the appropriate RECLAIM_COMPLETE.
5.3.2. Synchronizing Session Transfer 5.4.2. Synchronizing Session Transfer
When transferring state between the source and destination, the When transferring state between the source and destination, the
issues discussed in Section 7.2 of [RFC7931] must still be attended issues discussed in Section 7.2 of [RFC7931] must still be attended
to. In this case, the use of NFS4ERR_DELAY is still necessary in to. In this case, the use of NFS4ERR_DELAY is still necessary in
NFSv4.1, as it was in NFSv4.0, to prevent locking state changing NFSv4.1, as it was in NFSv4.0, to prevent locking state changing
while it is being transferred. while it is being transferred.
There are a number of important differences in the NFS4.1 context: There are a number of important differences in the NFS4.1 context:
o The absence of RELEASE_LOCKOWNER means that the one case in which o The absence of RELEASE_LOCKOWNER means that the one case in which
skipping to change at page 33, line 20 skipping to change at page 34, line 33
NFS4ERR_DELAY or NFS4ERR_MOVED until the client has established NFS4ERR_DELAY or NFS4ERR_MOVED until the client has established
the starting sequence for that slot on the destination server. the starting sequence for that slot on the destination server.
o Until the client has established the starting sequence for a o Until the client has established the starting sequence for a
particular slot on the destination server, do not report particular slot on the destination server, do not report
NFS4ERR_SEQ_MISORDERED or return a cached reply returning NFS4ERR_SEQ_MISORDERED or return a cached reply returning
NFS4ERR_DELAY or NFS4ERR_MOVED, where the reply consists solely of NFS4ERR_DELAY or NFS4ERR_MOVED, where the reply consists solely of
a series of operations where the response is NFS4_OK until the a series of operations where the response is NFS4_OK until the
final error. final error.
5.4. Defining Client Responsibilities for NFSv4.1 5.4.3. Server Issues Dealing with RECLAIM_COMPLETE
When Transparent State Migration is not available, servers can
provide a grace-period limited to a single file system, giving
clients the opportunity to reestablish their locks, originally held
on the source server, on the destination server, using the same
reclaim options normally used to recover from a server restart.
As part of that process, clients need to signal the end of their
contribution to the lock recovery process for a particular file
system transition by using the RECLAIM_COMPLETE operation described
in [RFC5661] specifying an rca_one_fs value of TRUE.
Since the publication of that document there have been a number of
developments regarding the handling of this form of RECLAIM_COMPLETE
that create issues that need to be addressed:
o The treatment of RECLAIM_COMPLETE in [RFC5661] was not as explicit
about the purpose of rca_one_fs as it might have been, leading to
some implementor confusion.
o Some clients, most likely those intending to access a single file
system, have issued RECLAIM_COMPLETE operations specifying an
rca_one_fs value of TRUE even where no file system migration event
has occurred. In so doing, such clients have essentially ignored
the REQUIREMENT that a client perform a RECLAIM_COMPLETE operation
specifying an rca_one_fs value of FALSE before obtaining any new
locks.
o Servers, in supporting such servers, may have accepted such
RECLAIM_COMPLETE operations and treated them as if they were done
with an rca_one_fs value of FALSE.
These developments, while troubling, do not raise any substantive
difficulty, if the servers do not support fs migration. However, to
enable file system migration to be implemented, some work must be
done to make the rca_one_fs useful, while maintaining necessary
compatibility with existing implementations.
o A new treatment of RECLAIM_COMPLETE is needed to eliminate the
ambiguities within the one contained in [RFC5661] and making it
clear that rca_one_fs is not to be misused as it has been.
o While servers "SHOULD NOT" accept erroneous uses of
RECLAIM_COMPLETE with rca_one_fs TRUE (i.e. those not connected to
file system migration), the consequence of doing so need to be
elucidated. As a result, servers which have no possibility of
being the destination in cases of fs migration will continue to be
able to do what they have been doing while it is likely that
others will have cease to accepting such RECLAIM_COMPLETE
operations in place of those with rca_one_fs set to FALSE.
5.5. Defining Client Responsibilities for NFSv4.1 Transitions
The subsections below discuss the responsibilities of the client in The subsections below discuss the responsibilities of the client in
dealing with transition to a new server (migration) and to use of new dealing with transition to a new server (migration) and to use of new
network addresses in accessing existing servers. network addresses in accessing existing servers.
5.4.1. Client Recovery from Migration Events 5.5.1. Client Recovery from Migration Events
When a file system is migrated, there a number of migration-related When a file system is migrated, there a number of migration-related
status indications with which clients need to deal: status indications with which clients need to deal:
o If an attempt is made to use or return a filehandle within a file o If an attempt is made to use or return a filehandle within a file
system that has been migrated away from the server on which it was system that has been migrated away from the server on which it was
previously available, the error NFS4ERR_MOVED is returned. previously available, the error NFS4ERR_MOVED is returned.
This condition continues on subsequent attempts to access the file This condition continues on subsequent attempts to access the file
system in question. The only way the client can avoid the error system in question. The only way the client can avoid the error
skipping to change at page 35, line 14 skipping to change at page 37, line 36
the necessary recovery directly, providing that it be done the necessary recovery directly, providing that it be done
asynchronously, or ensuring that it is already under way. asynchronously, or ensuring that it is already under way.
o All instances of the LEASE_MOVED indication should be dealt with o All instances of the LEASE_MOVED indication should be dealt with
asynchronously, in a migration discovery thread whose job is to asynchronously, in a migration discovery thread whose job is to
clear that indication by fetching the appropriate location clear that indication by fetching the appropriate location
attribute. Because this thread will only be fetching a location attribute. Because this thread will only be fetching a location
attribute and the fs_status attribute for the file systems attribute and the fs_status attribute for the file systems
referenced by the client, it cannot receive MOVED indications. referenced by the client, it cannot receive MOVED indications.
Some useful guidance regarding possible implementation of a Some useful guidance regarding possible implementation of a
migration discovery thread can be found in Section 5.4.2. migration discovery thread can be found in Section 5.5.2.
o When a migration discovery thread happens upon a migrated file o When a migration discovery thread happens upon a migrated file
system (i.e. not present and not a referral), the thread is likely system (i.e. not present and not a referral), the thread is likely
to have cleared one (out of an unknown number) of file systems to have cleared one (out of an unknown number) of file systems
whose migration needs to be responded to. The discovery thread whose migration needs to be responded to. The discovery thread
needs to schedule the appropriate migration recovery (as described needs to schedule the appropriate migration recovery (as described
in Section 5.4.3). This is necessary to ensure that migrated file in Section 5.5.3). This is necessary to ensure that migrated file
systems will be referenced on the destination server in order to systems will be referenced on the destination server in order to
avoid unnecessary lease expiration. avoid unnecessary lease expiration.
For many of the migrated file systems discovered in this way, the For many of the migrated file systems discovered in this way, the
client has not received any MOVED indication. In such cases, client has not received any MOVED indication. In such cases,
lease recovery needs to be scheduled but it should not interfere lease recovery needs to be scheduled but it should not interfere
with continuation of the migration discovery function. with continuation of the migration discovery function.
o When a migration discovery thread receives a LEASE_MOVED o When a migration discovery thread receives a LEASE_MOVED
indication, it takes no special action but continues its normal indication, it takes no special action but continues its normal
operation. On the other hand, if a LEASE_MOVED indication is not operation. On the other hand, if a LEASE_MOVED indication is not
received, it indicates that the thread has completed its work received, it indicates that the thread has completed its work
successfully. successfully.
5.4.2. The Migration Discovery Process 5.5.2. The Migration Discovery Process
As noted above, LEASE_MOVED indications are best dealt with in a As noted above, LEASE_MOVED indications are best dealt with in a
migration discovery thread. Because of this structure, migration discovery thread. Because of this structure,
o No action needs to be taken for such indications received by the o No action needs to be taken for such indications received by the
migration discovery threads, since continuation of that thread's migration discovery threads, since continuation of that thread's
work will address the issue. work will address the issue.
o For such indications received in other contexts, the generally o For such indications received in other contexts, the generally
appropriate response is to initiate or otherwise provide for the appropriate response is to initiate or otherwise provide for the
skipping to change at page 37, line 11 skipping to change at page 39, line 30
o Otherwise, if there is any record that other requests saw a o Otherwise, if there is any record that other requests saw a
LEASE_MOVED indication, that record is cleared and the LEASE_MOVED indication, that record is cleared and the
verification request retried. The discovery thread remains in verification request retried. The discovery thread remains in
completion/verification state. completion/verification state.
o If there has been no LEASE_MOVED indication, the work of the o If there has been no LEASE_MOVED indication, the work of the
discovery thread is considered completed and it enters the non- discovery thread is considered completed and it enters the non-
operating state. operating state.
5.4.3. Overview of Client Response to NFS4ERR_MOVED 5.5.3. Overview of Client Response to NFS4ERR_MOVED
This section outlines a way in which a client that receives This section outlines a way in which a client that receives
NFS4ERR_MOVED can respond by using a new server or network address if NFS4ERR_MOVED can respond by using a new server or network address if
one is available. As part of that process, it will determine: one is available. As part of that process, it will determine:
o Whether the NFS4ERR_MOVED indicates migration has occurred, or o Whether the NFS4ERR_MOVED indicates migration has occurred, or
whether it indicates another sort of file system transition as whether it indicates another sort of file system transition as
discussed in Section 5.2.2. discussed in Section 5.3.2.
o In the case of migration, whether Transparent State Migration has o In the case of migration, whether Transparent State Migration has
occurred. occurred.
o Whether any state has been lost during the process of Transparent o Whether any state has been lost during the process of Transparent
State Migration. State Migration.
o Whether sessions have been transferred as part of Transparent o Whether sessions have been transferred as part of Transparent
State Migration. State Migration.
skipping to change at page 38, line 10 skipping to change at page 40, line 29
EXCHANGE_ID results indicate that the current location element is EXCHANGE_ID results indicate that the current location element is
server-trunkable with that used to access the file system when server-trunkable with that used to access the file system when
access was terminated by receiving NFS4ERR_MOVED. If it is, then access was terminated by receiving NFS4ERR_MOVED. If it is, then
migration has not occurred and the transition is dealt with, at migration has not occurred and the transition is dealt with, at
least initially, as one involving continued access to the same least initially, as one involving continued access to the same
file system on the same server through a new network address. file system on the same server through a new network address.
3. Obtaining access to existing session state or creating new 3. Obtaining access to existing session state or creating new
sessions. How this is done depends on the initial determination sessions. How this is done depends on the initial determination
of whether migration has occurred and can be done as described in of whether migration has occurred and can be done as described in
Section 5.4.4 in the case of migration or as described in Section 5.5.4 in the case of migration or as described in
Section 5.4.5 in the case of a network address transfer without Section 5.5.5 in the case of a network address transfer without
migration. migration.
4. Verification of the trunking relationship assumed in step 2 as 4. Verification of the trunking relationship assumed in step 2 as
discussed in Section 2.10.5.1 of [RFC5661]. Although this step discussed in Section 2.10.5.1 of [RFC5661]. Although this step
will generally confirm the initial determination, it is possible will generally confirm the initial determination, it is possible
for verification to fail with the result that an initial for verification to fail with the result that an initial
determination that a network address shift (without migration) determination that a network address shift (without migration)
has occurred may be invalidated and migration determined to have has occurred may be invalidated and migration determined to have
occurred. There is no need to redo step 3 above, since it will occurred. There is no need to redo step 3 above, since it will
be possible to continue use of the session established already. be possible to continue use of the session established already.
5. Obtaining access to existing locking state and/or reobtaining it. 5. Obtaining access to existing locking state and/or reobtaining it.
How this is done depends on the final determination of whether How this is done depends on the final determination of whether
migration has occurred and can be done as described in migration has occurred and can be done as described in
Section 5.4.4 in the case of migration or as described in Section 5.5.4 in the case of migration or as described in
Section 5.4.5 in the case of a network address transfer without Section 5.5.5 in the case of a network address transfer without
migration. migration.
Once the initial address has been determined, clients are free to Once the initial address has been determined, clients are free to
apply an abbreviated process to find additional addresses trunkable apply an abbreviated process to find additional addresses trunkable
with it (clients may seek session-trunkable or server trunkable with it (clients may seek session-trunkable or server trunkable
addresses depending on whether they support clientid trunking). addresses depending on whether they support clientid trunking).
During this later phase of the process, further location entries are During this later phase of the process, further location entries are
examined using the abbreviated procedure specified below: examined using the abbreviated procedure specified below:
1. Before the EXCHANGE_ID, the fs_name field is examined and if it 1. Before the EXCHANGE_ID, the fs_name field is examined and if it
skipping to change at page 39, line 5 skipping to change at page 41, line 21
2. In the case that the network address is session-trunkable with 2. In the case that the network address is session-trunkable with
one used previously a BIND_CONN_TO_SESSION is used to access that one used previously a BIND_CONN_TO_SESSION is used to access that
session using new network address. Otherwise, or if the bind session using new network address. Otherwise, or if the bind
operation fails, a CREATE_SESSION is done. operation fails, a CREATE_SESSION is done.
3. The verification procedure referred to in step 4 above is used. 3. The verification procedure referred to in step 4 above is used.
However, if it fails, the entry is ignored and the next available However, if it fails, the entry is ignored and the next available
entry is used. entry is used.
5.4.4. Obtaining Access to Sessions and State after Migration 5.5.4. Obtaining Access to Sessions and State after Migration
In the event that migration has occurred, the determination of In the event that migration has occurred, the determination of
whether Transparent State Migration has occurred is driven by the whether Transparent State Migration has occurred is driven by the
client ID returned by the EXCHANGE_ID and the reported confirmation client ID returned by the EXCHANGE_ID and the reported confirmation
status. status.
o If the client ID is an unconfirmed client ID not previously known o If the client ID is an unconfirmed client ID not previously known
to the client, then Transparent State Migration has not occurred. to the client, then Transparent State Migration has not occurred.
o If the client ID is a confirmed client ID previously known to the o If the client ID is a confirmed client ID previously known to the
skipping to change at page 40, line 40 skipping to change at page 43, line 10
o In a case in which Transparent State Migration has occurred, and o In a case in which Transparent State Migration has occurred, and
some lock state was lost (as shown by SEQ4_STATUS flags), existing some lock state was lost (as shown by SEQ4_STATUS flags), existing
stateids need to be checked for validity using TEST_STATEID, and stateids need to be checked for validity using TEST_STATEID, and
reclaim used to re-establish any that were not transferred. reclaim used to re-establish any that were not transferred.
For all of the cases above, RECLAIM_COMPLETE with an rca_one_fs value For all of the cases above, RECLAIM_COMPLETE with an rca_one_fs value
of true should be done before normal use of the file system including of true should be done before normal use of the file system including
obtaining new locks for the file system. This applies even if no obtaining new locks for the file system. This applies even if no
locks were lost and needed to be reclaimed. locks were lost and needed to be reclaimed.
5.4.5. Obtaining Access to Sessions and State after Network Address 5.5.5. Obtaining Access to Sessions and State after Network Address
Transfer Transfer
The case in which there is a transfer to a new network address The case in which there is a transfer to a new network address
without migration is similar to that described in Section 5.4.4 in without migration is similar to that described in Section 5.5.4 in
that there is a need to obtain access to needed sessions and locking that there is a need to obtain access to needed sessions and locking
state. However, the details are simpler and will vary depending on state. However, the details are simpler and will vary depending on
the type of trunking between the address receiving NFS4ERR_MOVED and the type of trunking between the address receiving NFS4ERR_MOVED and
that to which the transfer is to be made that to which the transfer is to be made
To make a session available for use, a BIND_CONN_TO_SESSION should be To make a session available for use, a BIND_CONN_TO_SESSION should be
used to obtain access to the session previously in use. Only if this used to obtain access to the session previously in use. Only if this
fails, should a CREATE_SESSION be done. While this procedure mirrors fails, should a CREATE_SESSION be done. While this procedure mirrors
that in Section 5.4.4, there is an important difference in that that in Section 5.5.4, there is an important difference in that
preservation of the session is not purely optional but depends on the preservation of the session is not purely optional but depends on the
type of trunking. type of trunking.
Access to appropriate locking state should need no actions beyond Access to appropriate locking state should need no actions beyond
access to the session. However. the SEQ4_STATUS bits should be access to the session. However. the SEQ4_STATUS bits should be
checked for lost locking state, including the need to reclaim locks checked for lost locking state, including the need to reclaim locks
after a server reboot. after a server reboot.
5.5. Resolution of NFSv4.1 Issues 5.6. Resolution of NFSv4.1 Issues
One possibility is that addressing all of the NFSv4.1 issues would One possibility is that addressing all of the NFSv4.1 issues would
entail publication of a standards-track document updating [RFC5661]. entail publication of a standards-track document updating [RFC5661].
Such a document would have three major elements: Such a document would have three major elements:
o A considerable expansion of the existing Section 11.4 explaining o A considerable expansion of the existing Section 11.4 explaining
the various uses of the location attribute and the possible the various uses of the location attribute and the possible
interactions among these various uses. This, like the interactions among these various uses. This, like the
corresponding replacement section for NFSv4.0 would be based on corresponding replacement section for NFSv4.0 would be based on
our Section 3.2 above. Information regarding the specifics of our Section 3.2 above. Information regarding the specifics of
trunking discovery might appear in this section, in a new sub- trunking discovery might appear in this section, in a new sub-
section. As part of this revision, the existing Section 11.4.2 section. As part of this revision, the existing Section 11.4.2
would need to be revised to explain all the possible results of would need to be revised to explain all the possible results of
NFS4ERR_MOVED including migration and a possible transparent NFS4ERR_MOVED including migration and a possible transparent
transition in which the network address changes but the server transition in which the network address changes but the server
does not. does not.
o A revision of the existing section 18.35 (dealing with o A revision of the existing section 18.35 (dealing with
EXCHANGE_ID) addressing the issues discussed in Section 5.2.1. EXCHANGE_ID) addressing the issues discussed in Section 5.3.1.
o A major replacement of the existing Section 11.7, entitled o A major replacement of the existing Section 11.7, entitled
"Effecting File System Transitions", as discussed below. "Effecting File System Transitions", as discussed below.
In addition, there is a set of smaller changes necessary In addition, there is a set of smaller changes necessary
o Update the existing Section 2.10.5 to clarify the proper response o Update the existing Section 2.10.5 to clarify the proper response
to server_owner changes, as described in our Section 5.1.3. to server_owner changes, as described in our Section 5.1.2.
o Replacement of the existing Section 15.1.2.4 to reflect the fact o Replacement of the existing Section 15.1.2.4 to reflect the fact
that NFS4ERR_MOVED can occur when a file system is now accessible that NFS4ERR_MOVED can occur when a file system is now accessible
at a different network address. A possible replacement text might at a different network address. A possible replacement text might
be: be:
The file system that contains the current filehandle object is The file system that contains the current filehandle object is
not accessible using the network address which has been used. not accessible using the network address which has been used.
It may have been relocated, migrated to another server, be It may have been relocated, migrated to another server, be
accessible using another network address on the current server, accessible using another network address on the current server,
skipping to change at page 43, line 38 skipping to change at page 46, line 7
o Adding a new section (at level of the existing Section 11.7.7) o Adding a new section (at level of the existing Section 11.7.7)
about state transfer during migration. Although the phrase about state transfer during migration. Although the phrase
"Transparent State Migration" is well established in the context "Transparent State Migration" is well established in the context
of NFSv4.0, the word "transparent" could cause confusion given the of NFSv4.0, the word "transparent" could cause confusion given the
existing use of the phrase "transparent transitions". A possible existing use of the phrase "transparent transitions". A possible
title for the new section is "State Transfer during Migration" title for the new section is "State Transfer during Migration"
The new section would present the NFSv4.1-equivalent of Transparent The new section would present the NFSv4.1-equivalent of Transparent
State Migration as described in [RFC7931]. This would address the State Migration as described in [RFC7931]. This would address the
issues presented in Section 5.1 along the lines suggested in Sections issues presented in Section 5.2 along the lines suggested in Sections
5.2, 5.3, and 5.4. 5.3, 5.4, and 5.5.
5.6. Potential Protocol Extensions 5.7. Potential Protocol Extensions
There are a number possibilities to provides additional facilities There are a number possibilities to provides additional facilities
related to issues discussed in this document using the protocol related to issues discussed in this document using the protocol
extension mechanisms described in [RFC8178]. These facilities relate extension mechanisms described in [RFC8178]. These facilities relate
to the handling of multiple connection types. to the handling of multiple connection types.
The possibility of additional connection types was not addressed in The possibility of additional connection types was not addressed in
NFSv4.0, either in [RFC3530] or [RFC7530]. While the use of mutiple NFSv4.0, either in [RFC3530] or [RFC7530]. While the use of multiple
connection types is allowed, facilities to determine the connection connection types is allowed, facilities to determine the connection
type to be used are sub-optimal and are expected to remain so. type to be used are sub-optimal and are expected to remain so.
In the case of NFSv4.1, there are facilities to aid in the In the case of NFSv4.1, there are facilities to aid in the
determination of connection types that can be used. However, such determination of connection types that can be used. However, such
facilities are limited to the two connection types already defined facilities are limited to the two connection types already defined
and may have weaknesses in dealing with changes in the set of and may have weaknesses in dealing with changes in the set of
connection types to be used and in selecting connections to be used, connection types to be used and in selecting connections to be used,
particularly in clustered server environments, in which the set of particularly in clustered server environments, in which the set of
potential trunked server endpoints can be large. potential trunked server endpoints can be large.
In light of this situation, it appears that a number of potential In light of this situation, it appears that a number of potential
extensions to NFSv4 might be considered, as provided for by extensions to NFSv4 might be considered, as provided for by
[RFC8178]. Such extensions could take the form of additional [RFC8178]. Such extensions could take the form of additional
OPTIONAL attributes. While these attributes would be part of OPTIONAL attributes. While these attributes would be part of
NFSv4.2, the fact that there is no change in the set of REQUIRED NFSv4.2, the fact that there is no change in the set of REQUIRED
features between NFSv4.1 and NFSv4.2 means that the upgrade path for features between NFSv4.1 and NFSv4.2 means that the upgrade path for
clients and servers can be made relatively simple. clients and servers can be made relatively simple.
The additonal attributes sketched out below would provide a more The additional attributes sketched out below would provide a more
complete way of addressing the possibility of trunking of a large set complete way of addressing the possibility of trunking of a large set
of server endpoints, of multiple connection types: of server endpoints, of multiple connection types:
o A new fs-scoped attribute, fs_location_endpoints, could provide o A new fs-scoped attribute, fs_location_endpoints, could provide
potential locations of a file system by using location entries potential locations of a file system by using location entries
specifying each potential endpoint, rather than specifing, as do specifying each potential endpoint, rather than specifying, as do
fs_locations and fs_locations_info, the nework address applicable fs_locations and fs_locations_info, the network address applicable
to all potenia endpoints. to all potential endpoints.
o A new server-scoped attribute, server_endpoints, could provide a o A new server-scoped attribute, server_endpoints, could provide a
set of trunkable endpoints to be used to access the current set of trunkable endpoints to be used to access the current
server, together with additional performance-related information server, together with additional performance-related information
useful for endpoint selection. useful for endpoint selection.
A fuller elaboration of these proposals would require the writing of A fuller elaboration of these proposals would require the writing of
one or more standards-track documents, assuming sufficient interest one or more standards-track documents, assuming sufficient interest
in proceeding along this route. Any such work would be separate from in proceeding along this route. Any such work would be separate from
other work suggested to resolve existing protocol issues and will not other work suggested to resolve existing protocol issues and will not
skipping to change at page 46, line 31 skipping to change at page 49, line 6
6.2. Further Work Needed 6.2. Further Work Needed
The following table classifies issues in this area and indicates The following table classifies issues in this area and indicates
which are currently adequately addressed and where the protocol which are currently adequately addressed and where the protocol
specifications need further correction or clarification. Where the specifications need further correction or clarification. Where the
topic is adequately addressed, a reference is given to the RFC topic is adequately addressed, a reference is given to the RFC
providing support for the issue. In other cases, an area name providing support for the issue. In other cases, an area name
(explained below) is given. (explained below) is given.
+-------+-----------+----------+-----------+----------+-------------+ +-------+-----------+----------+-----------+----------+-------------+
| Vers. | Trunking | Trunking | Xparent | Multiple | Interaction | | Vers. | Trunking | Trunking | State | Multiple | Interaction |
| | Detection | Disc. | State | Conn. | of Trunking | | | Detection | Disc. | Migration | Conn. | of Trunking |
| | | | Migration | Types | and | | | | | | Types | and |
| | | | | | Migration | | | | | | | Migration |
+-------+-----------+----------+-----------+----------+-------------+ +-------+-----------+----------+-----------+----------+-------------+
| v4.0 | [RFC7931] | TrDisc-0 | [RFC7931] | Mct-0 | Int-0 | | v4.0 | [RFC7931] | TrDisc-0 | [RFC7931] | Mct-0 | Int-0 |
| v4.1+ | [RFC5661] | TrDisc-1 | TSM-1 | Mct-1 | Int-1 | | v4.1+ | [RFC5661] | TrDisc-1 | SM-1 | Mct-1 | Int-1 |
+-------+-----------+----------+-----------+----------+-------------+ +-------+-----------+----------+-----------+----------+-------------+
The following table explains the work that needs to be done The following table explains the work that needs to be done
corresponding to each area name above. corresponding to each area name above.
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| Area | Description | | Area | Description |
| Name | | | Name | |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| TrDisc-0 | Although it is possible for there to be multiple | | TrDisc-0 | Although it is possible for there to be multiple |
skipping to change at page 47, line 16 skipping to change at page 49, line 40
| | not happen. | | | not happen. |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| TrDisc-1 | Despite the fact that [RFC5661] provides a means of | | TrDisc-1 | Despite the fact that [RFC5661] provides a means of |
| | trunking detection, trunking discovery was not | | | trunking detection, trunking discovery was not |
| | addressed. This problem was compounded by confusion | | | addressed. This problem was compounded by confusion |
| | regarding multiple file system replicas arising from | | | regarding multiple file system replicas arising from |
| | the fact that multiple network addresses connected to | | | the fact that multiple network addresses connected to |
| | the same server were treated as if they were referring | | | the same server were treated as if they were referring |
| | to distinct sets of replicas. | | | to distinct sets of replicas. |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| TSM-1 | Unlike [RFC7530], which mishandled Transparent State | | SM-1 | Unlike [RFC7530], which mishandled Transparent State |
| | Migration because of confusion arising from the lack | | | Migration because of confusion arising from the lack |
| | of appropriate trunking support, [RFC5661] simply | | | of appropriate trunking support, [RFC5661] simply |
| | neglected to provide any description of this feature. | | | neglected to provide any description of this feature. |
| | It appears likely that confusion between the needs of | | | It appears likely that confusion between the needs of |
| | migration and those of dealing with shifts in | | | migration and those of dealing with shifts in |
| | responsibility for clustered file system access had | | | responsibility for clustered file system access had |
| | significant role in allowing this issue to be ignored. | | | significant role in allowing this issue to be ignored. |
| | Rectifying this situation along the lines of [RFC7931] | | | Rectifying this situation along the lines of [RFC7931] |
| | is complicated by the need to rewrite significant | | | is complicated by the need to rewrite significant |
| | pieces of the section about multi-server namespace to | | | pieces of the section about multi-server namespace to |
| | address this confusion. Beyond this, the necessary | | | address this confusion. Beyond this, the necessary |
| | treatment will need to reflect changes required by the | | | treatment will need to reflect changes required by the |
| | use of the sessions model and related changes in | | | use of the sessions model and related changes in |
| | NFSv.1 and also address migration-related issues | | | NFSv.1 and also address migration-related issues |
| | raised by optional features such as pNFS and the | | | raised by optional features such as pNFS and the |
| | fs_locations_info attribute. | | | fs_locations_info attribute. In addition to |
| | correcting the handling of Transparent State |
| | migration, work also needs to be done to address |
| | migration-related issues in the handling of |
| | RECLAIM_COMPLETE. |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| Mct-0 | Even though protocol support for multiple connection | | Mct-0 | Even though protocol support for multiple connection |
| | types is quite limited in NFSv4.0, there still are | | | types is quite limited in NFSv4.0, there still are |
| | multiple connection types specified and implemented. | | | multiple connection types specified and implemented. |
| | As a result, some guidance has to be given to allow | | | As a result, some guidance has to be given to allow |
| | interoperable implementations to be developed, and | | | interoperable implementations to be developed, and |
| | used, without extensive user configuration effort. | | | used, without extensive user configuration effort. |
| | This should include some treatment of situtions in | | | This should include some treatment of situations in |
| | which the set of connection types to be used to access | | | which the set of connection types to be used to access |
| | a given file system changes, requiring appropriate | | | a given file system changes, requiring appropriate |
| | recovery from an NFS4ERR_MOVED error. | | | recovery from an NFS4ERR_MOVED error. |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| Mct-1 | Even though protocol support for multiple connection | | Mct-1 | Even though protocol support for multiple connection |
| | types is more limited than one might like, there are | | | types is more limited than one might like, there are |
| | helpful facilities that can be used to simplify the | | | helpful facilities that can be used to simplify the |
| | process of determining the connection type(s) to be | | | process of determining the connection type(s) to be |
| | uused. The proper use of the available facilities | | | used. The proper use of the available facilities |
| | needs to be clarified including examination of cases | | | needs to be clarified including examination of cases |
| | in which the set of connection types to be used to | | | in which the set of connection types to be used to |
| | access a given file system changes, requiring | | | access a given file system changes, requiring |
| | appropriate recovery from an NFS4ERR_MOVED error. | | | appropriate recovery from an NFS4ERR_MOVED error. |
+----------+--------------------------------------------------------+ +----------+--------------------------------------------------------+
| Int-0 | The need to provide trunking-related information puts | | Int-0 | The need to provide trunking-related information puts |
| | additional focus on the issue of dealing with changes | | | additional focus on the issue of dealing with changes |
| | in the value of location-related attributes. This | | | in the value of location-related attributes. This |
| | applies when trunking configurations change and at | | | applies when trunking configurations change and at |
| | other times as well. In addition, the existence of | | | other times as well. In addition, the existence of |
skipping to change at page 48, line 40 skipping to change at page 51, line 19
specification of NFS version 4.0 or 4.1 would need to be familiar specification of NFS version 4.0 or 4.1 would need to be familiar
with a large set of RFCs. with a large set of RFCs.
o One could have a document addressing all of the areas above Such a o One could have a document addressing all of the areas above Such a
document would update both [RFC7530] and [RFC5661]. That would document would update both [RFC7530] and [RFC5661]. That would
result in a confusing document given how different the v4.0 and result in a confusing document given how different the v4.0 and
v4.1 protocols are, since most readers will want a clear v4.1 protocols are, since most readers will want a clear
description of one or the other. description of one or the other.
o It is also possible to produce separate documents addressing o It is also possible to produce separate documents addressing
Trdisc-*, TSM-1., Mct-*, and Int-*. This would be subject many of Trdisc-*, SM-1., Mct-*, and Int-*. This would be subject many of
the difficulties of the two approaches above. the difficulties of the two approaches above.
The alternative, of organizing the changes by minor version, is being The alternative, of organizing the changes by minor version, is being
actively pursued by work on following Standards Track working group actively pursued by work on following Standards Track working group
documents: documents:
o [I-D.ietf-nfsv4-mv0-trunking-update] addresses the issues within o [I-D.ietf-nfsv4-mv0-trunking-update] addresses the issues within
TrDisc-0 and Int-0 by providing updates to [RFC7530], the vast TrDisc-0 and Int-0 by providing updates to [RFC7530], the vast
majority of which are within Section 8 of that document. Work to majority of which are within Section 8 of that document. Work to
include Mct-0 needs to be added. include Mct-0 needs to be added.
o [I-D.ietf-nfsv4-mv1-msns-update] addresses the issues within o [I-D.ietf-nfsv4-mv1-msns-update] addresses the issues within
TrDisc-1, TSM-1, and Int-1 providing updates to [RFC5661], the TrDisc-1, SM-1, and Int-1 providing updates to [RFC5661], the vast
vast majority of which are within Section 11 of that document. majority of which are within Section 11 of that document. Work to
Work to include Mct-1 is underway. include Mct-1 is underway.
These two documents will require additional review and discussion These two documents will require additional review and discussion
before proceeding to publication as Proposed standards, updating before proceeding to publication as Proposed standards, updating
[RFC7530] and [RFC5661] respectively. [RFC7530] and [RFC5661] respectively.
If the working group decides to continue along this path, it may be If the working group decides to continue along this path, it may be
desirable to consolidate the changes currently specified in these desirable to consolidate the changes currently specified in these
documents. Currently, these document replace individual sub-sections documents. Currently, these document replace individual sub-sections
of Section 8 (of [RFC7530]) or Section 11 (of [RFC5661]). While this of Section 8 (of [RFC7530]) or Section 11 (of [RFC5661]). While this
is helpful in explaining what is changing and why, things might be is helpful in explaining what is changing and why, things might be
skipping to change at page 51, line 45 skipping to change at page 54, line 23
[RFC8267] Lever, C., "Network File System (NFS) Upper-Layer Binding [RFC8267] Lever, C., "Network File System (NFS) Upper-Layer Binding
to RPC-over-RDMA Version 1", RFC 8267, to RPC-over-RDMA Version 1", RFC 8267,
DOI 10.17487/RFC8267, October 2017, DOI 10.17487/RFC8267, October 2017,
<https://www.rfc-editor.org/info/rfc8267>. <https://www.rfc-editor.org/info/rfc8267>.
9.2. Informative References 9.2. Informative References
[I-D.ietf-nfsv4-mv0-trunking-update] [I-D.ietf-nfsv4-mv0-trunking-update]
Lever, C. and D. Noveck, "NFS version 4.0 Trunking Lever, C. and D. Noveck, "NFS version 4.0 Trunking
Update", draft-ietf-nfsv4-mv0-trunking-update-00 (work in Update", draft-ietf-nfsv4-mv0-trunking-update-01 (work in
progress), January 2018. progress), July 2018.
[I-D.ietf-nfsv4-mv1-msns-update] [I-D.ietf-nfsv4-mv1-msns-update]
Noveck, D. and C. Lever, "NFSv4.1 Update for Multi-Server Noveck, D. and C. Lever, "NFSv4.1 Update for Multi-Server
Namespace", draft-ietf-nfsv4-mv1-msns-update-00 (work in Namespace", draft-ietf-nfsv4-mv1-msns-update-01 (work in
progress), January 2018. progress), June 2018.
[RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., [RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R.,
Beame, C., Eisler, M., and D. Noveck, "Network File System Beame, C., Eisler, M., and D. Noveck, "Network File System
(NFS) version 4 Protocol", RFC 3530, DOI 10.17487/RFC3530, (NFS) version 4 Protocol", RFC 3530, DOI 10.17487/RFC3530,
April 2003, <https://www.rfc-editor.org/info/rfc3530>. April 2003, <https://www.rfc-editor.org/info/rfc3530>.
Acknowledgments Acknowledgments
The editor and authors of this document gratefully acknowledge the The editor and authors of this document gratefully acknowledge the
contributions of Trond Myklebust of Primary Data, Robert Thurlow of contributions of Trond Myklebust of Primary Data, Robert Thurlow of
Oracle, and Andy Adamson of NetApp. We also thank Tom Haynes of Oracle, and Andy Adamson of NetApp. We also thank Tom Haynes of
Primary Data and Spencer Shepler of Microsoft for their guidance and Primary Data and Spencer Shepler of Microsoft for their guidance and
suggestions. suggestions.
Rick Macklem provided an analysis of the current description of
RECLAIM_COMPLETE and information about its implemenation for which we
are grateful.
Special thanks go to members of the Oracle Solaris NFS team, Special thanks go to members of the Oracle Solaris NFS team,
especially Rick Mesta and James Wahlig who were then part of that especially Rick Mesta and James Wahlig who were then part of that
team, for their work implementing an NFSv4.0 migration prototype and team, for their work implementing an NFSv4.0 migration prototype and
identifying many of the issues documented here. Also, the work of identifying many of the issues documented here. Also, the work of
Xuan Qi for Oracle using NFSv4.1 client and server prototypes was Xuan Qi for Oracle using NFSv4.1 client and server prototypes was
helpful. helpful.
Authors' Addresses Authors' Addresses
David Noveck (editor) David Noveck (editor)
 End of changes. 100 change blocks. 
240 lines changed or deleted 355 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/