draft-ietf-nfsv4-mv1-msns-update-02.txt   draft-ietf-nfsv4-mv1-msns-update-03.txt 
NFSv4 D. Noveck, Ed. NFSv4 D. Noveck, Ed.
Internet-Draft NetApp Internet-Draft NetApp
Updates: 5661 (if approved) C. Lever Updates: 5661 (if approved) C. Lever
Intended status: Standards Track ORACLE Intended status: Standards Track ORACLE
Expires: April 24, 2019 October 21, 2018 Expires: May 17, 2019 November 13, 2018
NFS Version 4.1 Update for Multi-Server Namespace NFS Version 4.1 Update for Multi-Server Namespace
draft-ietf-nfsv4-mv1-msns-update-02 draft-ietf-nfsv4-mv1-msns-update-03
Abstract Abstract
This document presents necessary clarifications and corrections This document presents necessary clarifications and corrections
concerning features related to the use of location-related attributes concerning features related to the use of attributes in NFSv4.1
in NFSv4.1. These include migration, which transfers responsibility related to file system location. These features include migration,
for a file system from one server to another, and facilities to which transfers responsibility for a file system from one server to
support trunking by allowing discovery of the set of network another, and facilities to support trunking by allowing discovery of
addresses to use to access a file system. This document updates the set of network addresses to use to access a file system. This
RFC5661. document updates RFC5661.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 24, 2019. This Internet-Draft will expire on May 17, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Requirements Language . . . . . . . . . . . . . . . . . . . . 4 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 6
3. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 4 3. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5 3.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6
3.2. Summary of Issues . . . . . . . . . . . . . . . . . . . . 7 3.2. Summary of Issues . . . . . . . . . . . . . . . . . . . . 8
3.3. Relationship of this Document to RFC5661 . . . . . . . . 9 3.3. Relationship of this Document to [RFC5661] . . . . . . . 10
4. Changes to Section 11 of RFC5661 . . . . . . . . . . . . . . 10 4. Changes to Section 11 of [RFC5661] . . . . . . . . . . . . . 11
4.1. Multi-Server Namespace (as updated) . . . . . . . . . . . 11 4.1. Updated introductory material for Section 11 of [RFC5661]
4.2. Location-related Terminology (to be added) . . . . . . . 11 entitled "Multi-Server Namespace" . . . . . . . . . . . . 12
4.3. Location Attributes (as updated) . . . . . . . . . . . . 13 4.2. New section to be added as the first sub-section of
4.4. Re-organization of Sections 11.4 and 11.5 of RFC5661 . . 14 Section 11 of [RFC5661] to be entitled
4.5. Uses of Location Information (as updated) . . . . . . . . 14 "Terminology Related to File System Location" . . . . . . 12
4.5.1. Combining Multiple Uses in a Single Attribute (to be 4.3. Updated Section 11.1 of [RFC5661] to be retitled
added) . . . . . . . . . . . . . . . . . . . . . . . 15 "File System Location Attributes" . . . . . . . . . . . . 14
4.5.2. Location Attributes and Trunking (to be added) . . . 16 4.4. Re-organization of Sections 11.4 and 11.5 of [RFC5661] . 15
4.5.3. Location Attributes and Connection Type Selection (to 4.5. Updated Section 11.4 of [RFC5661] to be retitled
be added) . . . . . . . . . . . . . . . . . . . . . . 16 "Uses of File System Location Information" . . . . . . . 15
4.5.4. File System Replication (as updated) . . . . . . . . 17 4.5.1. New section to be added as the first sub-section of
4.5.5. File System Migration (as updated) . . . . . . . . . 17 Section 11.4 of [RFC5661] to be entitled
4.5.6. Referrals (as updated) . . . . . . . . . . . . . . . 19 "Combining Multiple Uses in a Single Attribute" . . . 16
4.5.7. Changes in a Location Attribute (to be added) . . . . 20 4.5.2. New section to be added as the second sub-section of
5. Re-organization of Section 11.7 of RFC5661 . . . . . . . . . 21 Section 11.4 of [RFC5661] to be entitled
6. Overview of File Access Transitions (to be added) . . . . . . 22 "File System Location Attributes and Trunking" . . . 17
7. Effecting Network Endpoint Transitions (to be added) . . . . 22 4.5.3. New section to be added as the third sub-section of
8. Effecting File System Transitions (as updated) . . . . . . . 23 Section 11.4 of [RFC5661] to be entitled
8.1. File System Transitions and Simultaneous Access (as "File System Location Attributes and Connection Type
updated) . . . . . . . . . . . . . . . . . . . . . . . . 24 Selection" . . . . . . . . . . . . . . . . . . . . . 18
8.2. Filehandles and File System Transitions (as updated) . . 24 4.5.4. Updated Section 11.4.1 of [RFC5661] entitled
8.3. Fileids and File System Transitions (as updated) . . . . 25 "File System Replication" . . . . . . . . . . . . . . 19
8.4. Fsids and File System Transitions (as updated) . . . . . 26 4.5.5. Updated Section 11.4.2 of [RFC5661] entitled
8.4.1. File System Splitting (as updated) . . . . . . . . . 26 "File System Migration" . . . . . . . . . . . . . . . 19
8.5. The Change Attribute and File System Transitions (as 4.5.6. Updated Section 11.4.3 of [RFC5661] entitled
updated) . . . . . . . . . . . . . . . . . . . . . . . . 27 "Referrals" . . . . . . . . . . . . . . . . . . . . . 20
8.6. Write Verifiers and File System Transitions (as updated) 27 4.5.7. New section to be added after Section 11.4.3 of
8.7. Readdir Cookies and Verifiers and File System Transitions [RFC5661] to be entitled
(as updated) . . . . . . . . . . . . . . . . . . . . . . 28 "Changes in a File System Location Attribute" . . . . 22
8.8. File System Data and File System Transitions (as updated) 28 5. Re-organization of Section 11.7 of [RFC5661] . . . . . . . . 23
8.9. Lock State and File System Transitions (as updated) . . . 29 6. New section to be added after Section 11.6 of [RFC5661]
9. Transferring State upon Migration (to be added) . . . . . . . 30 to be entitled "Overview of File Access Transitions" . . . . 23
9.1. Transparent State Migration and pNFS (to be added) . . . 31 7. New section to be added second after Section 11.6 of
10. Client Responsibilities when Access is Transitioned (to be [RFC5661] to be entitled
added) . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 "Effecting Network Endpoint Transitions" . . . . . . . . . . 24
10.1. Client Transition Notifications (to be added) . . . . . 32 8. Updated Section 11.7 of [RFC5661] entitled
10.2. Performing Migration Discovery (to be added) . . . . . . 35 "Effecting File System Transitions" . . . . . . . . . . . . . 25
10.3. Overview of Client Response to NFS4ERR_MOVED (to be 8.1. Updated Section 11.7.1 of [RFC5661] entitled
added) . . . . . . . . . . . . . . . . . . . . . . . . . 37 "File System Transitions and Simultaneous Access" . . . . 25
10.4. Obtaining Access to Sessions and State after Migration 8.2. Updated Section 11.7.3 of [RFC5661] entitled
(to be added) . . . . . . . . . . . . . . . . . . . . . 39 "Filehandles and File System Transitions" . . . . . . . . 26
10.5. Obtaining Access to Sessions and State after Network 8.3. Updated Section 11.7.4 of [RFC5661] entitled
Address Transfer (to be added) . . . . . . . . . . . . . 41 "Fileids and File System Transitions" . . . . . . . . . . 27
11. Server Responsibilities Upon Migration (to be added) . . . . 41 8.4. Updated section 11.7.5 of [RFC5661] entitled
11.1. Server Responsibilities in Effecting State Reclaim after "Fsids and File System Transitions" . . . . . . . . . . . 28
Migration (to be added) . . . . . . . . . . . . . . . . 42 8.4.1. Updated section 11.7.5.1 of [RFC5661] entitled
11.2. Server Responsibilities in Effecting Transparent State "File System Splitting" . . . . . . . . . . . . . . . 28
Migration (to be added) . . . . . . . . . . . . . . . . 42 8.5. Updated Section 11.7.6 of [RFC5661] entitled
11.3. Server Responsibilities in Effecting Session Transfer "The Change Attribute and File System Transitions" . . . 29
(to be added) . . . . . . . . . . . . . . . . . . . . . 44 8.6. Updated Section 11.7.8 of [RFC5661] entitled
12. fs_locations_info . . . . . . . . . . . . . . . . . . . . . . 46 "Write Verifiers and File System Transitions" . . . . . . 29
12.1. Updates to treatment of fs_locations_info . . . . . . . 47 8.7. Updated Section 11.7.9 of [RFC5661] entitled
12.2. The Attribute fs_locations_info (as updated) . . . . . . 47 "Readdir Cookies and Verifiers and File System
12.2.1. The fs_locations_server4 Structure (as updated) . . 51 Transitions)" . . . . . . . . . . . . . . . . . . . . . . 29
12.2.2. The fs_locations_info4 Structure (as updated) . . . 57 8.8. Updated Section 11.7.10 of [RFC5661] entitled
12.2.3. The fs_locations_item4 Structure (as updated) . . . 59 "File System Data and File System Transitions" . . . . . 30
13. Changes to RFC5661 outside Section 11 . . . . . . . . . . . . 61 8.9. Updated Section 11.7.7 of [RFC5661] entitled
13.1. (Introduction to) Multi-Server Namespace (as updated) . 62 "Lock State and File System Transitions" . . . . . . . . 31
13.2. Server Scope (as updated) . . . . . . . . . . . . . . . 62 9. New section to be added after Section 11.11 of [RFC5661]
13.3. Revised Treatment of NFS4ERR_MOVED . . . . . . . . . . . 64 to be entitled "Transferring State upon Migration" . . . . . 32
13.4. Revised Discussion of Server_owner changes . . . . . . . 65 9.1. Only sub-section within new section to be added to
13.5. Revision to Treatment of EXCHANGE_ID . . . . . . . . . . 65 [RFC5661] to be entitled
13.6. Revision to Treatment of RECLAIM_COMPLETE . . . . . . . 67 "Transparent State Migration and pNFS" . . . . . . . . . 32
13.7. Reclaim Errors (as updated) . . . . . . . . . . . . . . 67 10. New section to be added second after Section 11.11 of
13.7.1. NFS4ERR_COMPLETE_ALREADY (as updated; Error Code [RFC5661] to be entitled
10054) . . . . . . . . . . . . . . . . . . . . . . . 67 "Client Responsibilities when Access is Transitioned" . . . . 34
13.7.2. NFS4ERR_GRACE (as updated; Error Code 10013) . . . . 67 10.1. First sub-section within new section to be added to
13.7.3. NFS4ERR_NO_GRACE (as updated; Error Code 10033) . . 67 [RFC5661] to be entitled
13.7.4. NFS4ERR_RECLAIM_BAD (as updated; Error Code 10034) . 68 "Client Transition Notifications" . . . . . . . . . . . 34
13.7.5. NFS4ERR_RECLAIM_CONFLICT (as updated; Error Code 10.2. Second sub-section within new section to be added to
10035) . . . . . . . . . . . . . . . . . . . . . . . 68 [RFC5661] to be entitled
14. Operation 42: EXCHANGE_ID - Instantiate Client ID (as "Performing Migration Discovery" . . . . . . . . . . . . 37
updated) . . . . . . . . . . . . . . . . . . . . . . . . . . 68 10.3. Third sub-section within new section to be added to
15. Operation 58: RECLAIM_COMPLETE - Indicates Reclaims Finished [RFC5661] to be entitled
(as updated) . . . . . . . . . . . . . . . . . . . . . . . . 86 "Overview of Client Response to NFS4ERR_MOVED" . . . . . 39
16. Security Considerations . . . . . . . . . . . . . . . . . . . 90 10.4. Fourth sub-section within new section to be added to
17. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 92 [RFC5661] to be entitled
18. References . . . . . . . . . . . . . . . . . . . . . . . . . 92 "Obtaining Access to Sessions and State after Migration" 41
18.1. Normative References . . . . . . . . . . . . . . . . . . 92 10.5. Fifth sub-section within new section to be added to
18.2. Informative References . . . . . . . . . . . . . . . . . 93 [RFC5661] to be entitled
Appendix A. Classification of Document Sections . . . . . . . . 94 "Obtaining Access to Sessions and State after Network
Appendix B. Updates to RFC5661 . . . . . . . . . . . . . . . . . 95 Address Transfer" . . . . . . . . . . . . . . . . . . . 43
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 98 11. New section to be added third after Section 11.11 of
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 99
[RFC5661] to be entitled
"Server Responsibilities Upon Migration" . . . . . . . . . . 43
11.1. First sub-section within new section to be added to
[RFC5661] to be entitled
"Server Responsibilities in Effecting State Reclaim
after Migration" . . . . . . . . . . . . . . . . . . . . 44
11.2. Second sub-section within new section to be added to
[RFC5661] to be entitled
"Server Responsibilities in Effecting Transparent State
Migration" . . . . . . . . . . . . . . . . . . . . . . . 45
11.3. Third sub-section within new section to be added to
[RFC5661] to be entitled
"Server Responsibilities in Effecting Session Transfer" 46
12. fs_locations_info . . . . . . . . . . . . . . . . . . . . . . 49
12.1. Updates to treatment of fs_locations_info . . . . . . . 49
12.2. Updated Section 11.10 of [RFC5661] entitled
"The Attribute fs_locations_info" . . . . . . . . . . . 49
12.2.1. Updated section 11.10.1 of [RFC5661] entitled
"The fs_locations_server4 Structure" . . . . . . . . 53
12.2.2. Updated Section 11.10.2 of [RFC5661] entitled
"The fs_locations_info4 Structure" . . . . . . . . . 60
12.2.3. Updated Section 11.10.3 of [RFC5661] entitled
"The fs_locations_item4 Structure" . . . . . . . . . 61
13. Changes to [RFC5661] outside Section 11 . . . . . . . . . . . 63
13.1. Updated section 1.7.3.3 of [RFC5661] to be retitled
"Introduction to Multi-Server Namespace" . . . . . . . . 64
13.2. Updated Section 2.10.4 of [RFC5661] entitled
"Server Scope" . . . . . . . . . . . . . . . . . . . . . 65
13.3. Revised Treatment of NFS4ERR_MOVED . . . . . . . . . . . 66
13.4. Revised Discussion of Server_owner changes . . . . . . . 67
13.5. Revision to Treatment of EXCHANGE_ID . . . . . . . . . . 68
13.6. Revision to Treatment of RECLAIM_COMPLETE . . . . . . . 69
13.7. Updated Section 15.1.9 of [RFC5661] entitled
"Reclaim Errors" . . . . . . . . . . . . . . . . . . . . 70
13.7.1. Updated Section 15.1.9.1 of [RFC5661] entitled
"NFS4ERR_COMPLETE_ALREADY (Error Code 10054)" . . . 70
13.7.2. Updated Section 15.1.9.2 of [RFC5661] entitled
"NFS4ERR_GRACE (Error Code 10013)" . . . . . . . . . 70
13.7.3. Updated Section 15.1.9.3 of [RFC5661] entitled
"NFS4ERR_NO_GRACE (Error Code 10033)" . . . . . . . 70
13.7.4. Updated Section 15.1.9.4 of [RFC5661] entitled
"NFS4ERR_RECLAIM_BAD (Error Code 10034)" . . . . . . 70
13.7.5. Updated Section 15.1.9.5 of [RFC5661] entitled
"NFS4ERR_RECLAIM_CONFLICT (Error Code 10035)" . . . 71
14. Updated Section 18.35 of [RFC5661] entitled
"Operation 42: EXCHANGE_ID - Instantiate Client ID" . . . . . 71
15. Updated Section 18.51 of [RFC5661] entitled
"Operation 58: RECLAIM_COMPLETE - Indicates Reclaims
Finished" . . . . . . . . . . . . . . . . . . . . . . . . . . 89
16. Security Considerations . . . . . . . . . . . . . . . . . . . 93
17. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 95
18. References . . . . . . . . . . . . . . . . . . . . . . . . . 95
18.1. Normative References . . . . . . . . . . . . . . . . . . 95
18.2. Informative References . . . . . . . . . . . . . . . . . 96
Appendix A. Classification of Document Sections . . . . . . . . 97
Appendix B. Updates to [RFC5661] . . . . . . . . . . . . . . . . 98
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 102
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 102
1. Introduction 1. Introduction
This document defines the proper handling, within NFSv4.1, of the This document defines the proper handling, within NFSv4.1, of the
location-related attributes fs_locations and fs_locations_info and attributes related to file system location fs_locations and
how necessary changes in those attributes are to be dealt with. The fs_locations_info and how necessary changes in those attributes are
necessary corrections and clarifications parallel those done for to be dealt with. The necessary corrections and clarifications
NFSv4.0 in [RFC7931] and [I-D.cel-nfsv4-mv0-trunking-update]. parallel those done for NFSv4.0 in [RFC7931] and
[I-D.cel-nfsv4-mv0-trunking-update].
A large part of the changes to be made are necessary to clarify the A large part of the changes to be made are necessary to clarify the
handling of Transparent State Migration in NFSv4.1, which was omitted handling of Transparent State Migration in NFSv4.1, which was omitted
in [RFC5661]. Many of the issues dealt with in [RFC7931] need to be in [RFC5661]. Many of the issues dealt with in [RFC7931] need to be
addressed in the context of NFSv4.1. addressed in the context of NFSv4.1.
Another important issue to be dealt with concerns the handling of Another important issue to be dealt with concerns the handling of
multiple entries within location-related attributes that represent multiple entries within attributes related to file system locations
different ways to access the same file system. Unfortunately that represent different ways to access the same file system.
[RFC5661], while recognizing that these entries can represent Unfortunately [RFC5661], while recognizing that these entries can
different ways to access the same file system, confuses the matter by represent different ways to access the same file system, confuses the
treating network access paths as "replicas", making it difficult for matter by treating network access paths as "replicas", making it
these attributes to be used to obtain information about the network difficult for these attributes to be used to obtain information about
addresses to be used to access particular file system instances and the network addresses to be used to access particular file system
engendering confusion between two different sorts of transition: instances and engendering confusion between two different sorts of
those involving a change of network access paths to the same file transition: those involving a change of network access paths to the
system instance and those in which there is a shift between two same file system instance and those in which there is a shift between
distinct replicas. two distinct replicas.
When location information is used to determine the set of network When file system location information is used to determine the set of
addresses to access a particular file system instance (i.e. to network addresses to access a particular file system instance (i.e.
perform trunking discovery), clarification is needed regarding the to perform trunking discovery), clarification is needed regarding the
interaction of trunking and transitions between file system replicas, interaction of trunking and transitions between file system replicas,
including migration. Unfortunately [RFC5661], while it provided a including migration. Unfortunately [RFC5661], while it provided a
method of determining whether two network addresses were connected to method of determining whether two network addresses were connected to
the same server, did not address the issue of trunking discovery, the same server, did not address the issue of trunking discovery,
making it necessary to address it in this document. making it necessary to address it in this document.
2. Requirements Language 2. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
skipping to change at page 5, line 32 skipping to change at page 6, line 41
4.1. 4.1.
o Trunking detection refers to ways of deciding whether two specific o Trunking detection refers to ways of deciding whether two specific
network addresses are connected to the same NFSv4 server. The network addresses are connected to the same NFSv4 server. The
means available to make this determination depends on the protocol means available to make this determination depends on the protocol
version, and, in some cases, on the client implementation. version, and, in some cases, on the client implementation.
In the case of NFS version 4.1 and later minor versions, the means In the case of NFS version 4.1 and later minor versions, the means
of trunking detection are as described by [RFC5661] and are of trunking detection are as described by [RFC5661] and are
available to every client. Two network addresses connected to the available to every client. Two network addresses connected to the
same server are always server-trunkable but are not necessarily same server are always server-trunkable but cannot necessarily be
session-trunkable. used together to access a single session.
o Trunking discovery is a process by which a client using one o Trunking discovery is a process by which a client using one
network address can obtain other addresses that are connected to network address can obtain other addresses that are connected to
the same server. Typically it builds on a trunking detection the same server. Typically it builds on a trunking detection
facility by providing one or more methods by which candidate facility by providing one or more methods by which candidate
addresses are made available to the client who can then use addresses are made available to the client who can then use
trunking detection to appropriately filter them. trunking detection to appropriately filter them.
Despite the support for trunking detection there was no Despite the support for trunking detection there was no
description of trunking discovery provided in [RFC5661]. description of trunking discovery provided in [RFC5661].
skipping to change at page 6, line 26 skipping to change at page 7, line 34
over-RDMA as described in [RFC8166]. over-RDMA as described in [RFC8166].
o The combination of a server network address and a particular o The combination of a server network address and a particular
connection type to be used by a connection is referred to as a connection type to be used by a connection is referred to as a
"server endpoint". Although using different connection types may "server endpoint". Although using different connection types may
result in different ports being used, the use of different ports result in different ports being used, the use of different ports
by multiple connections to the same network address is not the by multiple connections to the same network address is not the
essence of the distinction between the two endpoints used. essence of the distinction between the two endpoints used.
o Two network addresses connected to the same server are said to be o Two network addresses connected to the same server are said to be
server-trunkable. server-trunkable. Two such addresses support the use of clientid
ID trunking, as described in [RFC5661]
o Two network addresses connected to the same server such that those o Two network addresses connected to the same server such that those
addresses can be used to support a single common session are addresses can be used to support a single common session are
referred to as session-trunkable. Note that two addresses may be referred to as session-trunkable. Note that two addresses may be
server-trunkable without being session-trunkable and that when two server-trunkable without being session-trunkable and that when two
connections of different connection types are made to the same connections of different connection types are made to the same
network address and are based on a single-location entry they are network address and are based on a single file system location
always session-trunkable, independent of the connection type, as entry they are always session-trunkable, independent of the
specified by [RFC5661], since their derivation from the same connection type, as specified by [RFC5661], since their derivation
location entry assures that both connections are to the same from the same file system location entry together with the
server. identity of their network addresses assures that both connections
are to the same server and will return server-owner information
allowing session trunking to be used.
Discussion of the term "replica" is complicated for a number of Discussion of the term "replica" is complicated for a number of
reasons: reasons:
o Even though the term is used in explaining the issues in [RFC5661] o Even though the term is used in explaining the issues in [RFC5661]
that need to be addressed in this document, a full explanation of that need to be addressed in this document, a full explanation of
this term requires explanation of related terms connected to the this term requires explanation of related terms connected to the
location attributes which are provided in Section 4.2 of the file system location attributes which are provided in Section 4.2
current document. of the current document.
o The term is also used in [RFC5661], with a meaning different from o The term is also used in [RFC5661], with a meaning different from
that in the current document. In short, in [RFC5661] each replica that in the current document. In short, in [RFC5661] each replica
is a identified by a single network access path while, in the is a identified by a single network access path while, in the
current document a set of network access paths which have server- current document a set of network access paths which have server-
trunkable network addresses and the same root-relative file system trunkable network addresses and the same root-relative file system
pathname are considered to be a single replica with multiple pathname are considered to be a single replica with multiple
network access paths. network access paths.
3.2. Summary of Issues 3.2. Summary of Issues
skipping to change at page 7, line 32 skipping to change at page 8, line 43
o [RFC5661], while it dealt with situations in which various forms o [RFC5661], while it dealt with situations in which various forms
of clustering allowed co-ordination of the state assigned by co- of clustering allowed co-ordination of the state assigned by co-
operating servers to be used, made no provisions for Transparent operating servers to be used, made no provisions for Transparent
State Migration, as introduced by [RFC7530] and corrected and State Migration, as introduced by [RFC7530] and corrected and
clarified by [RFC7931]. clarified by [RFC7931].
o Although NFSv4.1 was defined with a clear definition of how o Although NFSv4.1 was defined with a clear definition of how
trunking detection was to be done, there was no clear trunking detection was to be done, there was no clear
specification of how trunking discovery was to be done, despite specification of how trunking discovery was to be done, despite
the fact that the specification clearly indicated that this the fact that the specification clearly indicated that this
information could be made available via the location attributes. information could be made available via the file system location
attributes.
o Because the existence of multiple network access paths to the same o Because the existence of multiple network access paths to the same
file system was dealt with as if there were multiple replicas, file system was dealt with as if there were multiple replicas,
issues relating to transitions between replicas could never be issues relating to transitions between replicas could never be
clearly distinguished from trunking-related transitions between clearly distinguished from trunking-related transitions between
the addresses used to access a particular file system instance. the addresses used to access a particular file system instance.
As a result, in situations in which both migration and trunking As a result, in situations in which both migration and trunking
configuration changes were involved, neither of these could be configuration changes were involved, neither of these could be
clearly dealt with and the relationship between these two features clearly dealt with and the relationship between these two features
was not seriously addressed. was not seriously addressed.
skipping to change at page 9, line 21 skipping to change at page 10, line 28
the existing treatment of client id confirmation does not make the existing treatment of client id confirmation does not make
sense in the context of transparent state migration, in which sense in the context of transparent state migration, in which
client ids are transferred between source and destination servers. client ids are transferred between source and destination servers.
o A new treatment of RECLAIM_COMPLETE is needed, replacing that o A new treatment of RECLAIM_COMPLETE is needed, replacing that
which appeared in Section 18.51 of [RFC5661]. This is necessary which appeared in Section 18.51 of [RFC5661]. This is necessary
to clarify the function of the one-fs flag and clarify how to clarify the function of the one-fs flag and clarify how
existing clients, that might not properly use this flag, are to be existing clients, that might not properly use this flag, are to be
dealt with. dealt with.
3.3. Relationship of this Document to RFC5661 3.3. Relationship of this Document to [RFC5661]
The role of this document is to explain and specify a set of needed The role of this document is to explain and specify a set of needed
changes to [RFC5661]. All of these changes are related to the multi- changes to [RFC5661]. All of these changes are related to the multi-
server namespace features of NFSv4.1. server namespace features of NFSv4.1.
This document contains sections that propose additions to and other This document contains sections that propose additions to and other
modifications of [RFC5661] as well as others that explain the reasons modifications of [RFC5661] as well as others that explain the reasons
for modifications but do not directly affect existing specifications. for modifications but do not directly affect existing specifications.
In consequence, the sections of this document can be divided into In consequence, the sections of this document can be divided into
skipping to change at page 9, line 45 skipping to change at page 11, line 4
time as a consolidated specification is produced. time as a consolidated specification is produced.
o Explanatory sections do not contain any material that is meant to o Explanatory sections do not contain any material that is meant to
update the specification of NFSv4.1. Such sections may contain update the specification of NFSv4.1. Such sections may contain
explanations about why and how changes are to be done, without explanations about why and how changes are to be done, without
including any text that is to update [RFC5661] or appear in an including any text that is to update [RFC5661] or appear in an
eventual consolidated document, eventual consolidated document,
o Replacement sections contain text that is to replace and thus o Replacement sections contain text that is to replace and thus
supersede text within [RFC5661] and then appear in an eventual supersede text within [RFC5661] and then appear in an eventual
consolidated document. Replacement sections have the phrase "(as consolidated document. The titles of replacement sections
updated)" appended to the section title. indicate the section(s) within [RFC5661] that is to be replaced.
o Additional sections contain text which, although not replacing o Additional sections contain text which, although not replacing
anything in [RFC5661], will be part of the specification of anything in [RFC5661], will be part of the specification of
NFSv4.1 and will be expected to be part of an eventual NFSv4.1 and will be expected to be part of an eventual
consolidated document. Additional sections have the phrase "(to consolidated document. The titles of additional sections indicate
be added)" appended to the section title. where, within [RFC5661], the new section would appear.
o Editing sections contain some text that replaces text within o Editing sections contain some text that replaces text within
[RFC5661], although the entire section will not consist of such [RFC5661], although the entire section will not consist of such
text and will include other text as well. Such sections make text and will include other text as well. Such sections make
relatively minor adjustments in the existing NFSv4.1 specification relatively minor adjustments in the existing NFSv4.1 specification
which are expected to reflected in an eventual consolidated which are expected to reflected in an eventual consolidated
document. Generally such replacement text appears as a quotation, document. Generally such replacement text appears as a quotation,
which may take the form of an indented set of paragraphs. which may take the form of an indented set of paragraphs.
See Appendix A for a classification of the sections of this document See Appendix A for a classification of the sections of this document
according to the categories above. according to the categories above.
When this document is approved and published, [RFC5661] would be When this document is approved and published, [RFC5661] would be
significantly updated with most of the changed sections within the significantly updated with most of the changed sections within the
current Section 11 of that document. A detailed discussion of the current Section 11 of that document. A detailed discussion of the
necessary updates can be found in Appendix B. necessary updates can be found in Appendix B.
4. Changes to Section 11 of RFC5661 4. Changes to Section 11 of [RFC5661]
A number of sections need to be revised, replacing existing sub- A number of sections need to be revised, replacing existing sub-
sections within section 11 of [RFC5661]: sections within section 11 of [RFC5661]:
o New introductory material, including a terminology section, o New introductory material, including a terminology section,
replaces the existing material in [RFC5661] ranging from the start replaces the existing material in [RFC5661] ranging from the start
of the existing Section 11 up to and including the existing of the existing Section 11 up to and including the existing
Section 11.1. The new material appears in Sections 4.1 through Section 11.1. The new material appears in Sections 4.1 through
4.3 below. 4.3 below.
o A significant reorganization of the material in the existing o A significant reorganization of the material in the existing
Sections 11.4 and 11.5 (of [RFC5661]) is necessary. The reasons Sections 11.4 and 11.5 (of [RFC5661]) is necessary. The reasons
for the reorganization of these sections into a single section for the reorganization of these sections into a single section
with multiple subsections are discussed in Section 4.4 below. with multiple subsections are discussed in Section 4.4 below.
This replacement appears as Section 4.5 below. This replacement appears as Section 4.5 below.
New material relating to the handling of the location attributes New material relating to the handling of the file system location
is contained in Sections 4.5.1 and 4.5.7 below. attributes is contained in Sections 4.5.1 and 4.5.7 below.
o A major replacement for the existing Section 11.7 of [RFC5661] o A major replacement for the existing Section 11.7 of [RFC5661]
entitled "Effecting File System Transitions", will appear as entitled "Effecting File System Transitions", will appear as
Sections 6 through 11 of the current document. The reasons for Sections 6 through 11 of the current document. The reasons for
the reorganization of this section into multiple sections are the reorganization of this section into multiple sections are
discussed below in Section 5 of the current document. discussed below in Section 5 of the current document.
o A replacement for the existing Section 11.10 of [RFC5661] entitled o A replacement for the existing Section 11.10 of [RFC5661] entitled
"The Attribute fs_locations_info", will appear as Section 12.2 of "The Attribute fs_locations_info", will appear as Section 12.2 of
the current document, with Section 12.1 describing the differences the current document, with Section 12.1 describing the differences
between the new section and the treatment within [RFC5661]. A between the new section and the treatment within [RFC5661]. A
revised treatment is necessary because the existing treatment did revised treatment is necessary because the existing treatment did
not make clear how the added attribute information relates to the not make clear how the added attribute information relates to the
case of trunked paths to the same replica. These issues were not case of trunked paths to the same replica. These issues were not
addressed in [RFC5661] where the concepts of a replica and a addressed in [RFC5661] where the concepts of a replica and a
network path used to access a replica were not clearly network path used to access a replica were not clearly
distinguished. distinguished.
4.1. Multi-Server Namespace (as updated) 4.1. Updated introductory material for Section 11 of [RFC5661] entitled
"Multi-Server Namespace"
NFSv4.1 supports attributes that allow a namespace to extend beyond NFSv4.1 supports attributes that allow a namespace to extend beyond
the boundaries of a single server. It is desirable that clients and the boundaries of a single server. It is desirable that clients and
servers support construction of such multi-server namespaces. Use of servers support construction of such multi-server namespaces. Use of
such multi-server namespaces is OPTIONAL however, and for many such multi-server namespaces is OPTIONAL however, and for many
purposes, single-server namespaces are perfectly acceptable. Use of purposes, single-server namespaces are perfectly acceptable. Use of
multi-server namespaces can provide many advantages, by separating a multi-server namespaces can provide many advantages, by separating a
file system's logical position in a namespace from the (possibly file system's logical position in a namespace from the (possibly
changing) logistical and administrative considerations that result in changing) logistical and administrative considerations that result in
particular file systems being located on particular servers. particular file systems being located on particular servers.
4.2. Location-related Terminology (to be added) 4.2. New section to be added as the first sub-section of Section 11 of
[RFC5661] to be entitled "Terminology Related to File System
Location"
Regarding terminology relating to the construction of multi-server Regarding terminology relating to the construction of multi-server
namespaces out of a set of local per-server namespaces: namespaces out of a set of local per-server namespaces:
o Each server has a set of exported file systems which may accessed o Each server has a set of exported file systems which may accessed
by NFSv4 clients. Typically, this is done by assigning each file by NFSv4 clients. Typically, this is done by assigning each file
system a name within the pseudo-fs associated with the server, system a name within the pseudo-fs associated with the server,
although the pseudo-fs may be dispensed with if there is only a although the pseudo-fs may be dispensed with if there is only a
single exported file system. Each such file system is part of the single exported file system. Each such file system is part of the
server's local namespace, and can be considered as a file system server's local namespace, and can be considered as a file system
instance within a larger multi-server namespace. instance within a larger multi-server namespace.
o The set of all exported file systems for a given server o The set of all exported file systems for a given server
constitutes that server's local namespace. constitutes that server's local namespace.
o In some cases, a server will have a namespace more extensive than o In some cases, a server will have a namespace more extensive than
its local namespace, by using features associated with attributes its local namespace, by using features associated with attributes
that provide location information. These features, which allow that provide file system location information. These features,
construction of a multi-server namespace are all described in which allow construction of a multi-server namespace are all
individual sections below and include referrals (described in described in individual sections below and include referrals
Section 4.5.6), migration (described in Section 4.5.5), and (described in Section 4.5.6), migration (described in
replication (described in Section 4.5.4). Section 4.5.5), and replication (described in Section 4.5.4).
o A file system present in a server's pseudo-fs may have multiple o A file system present in a server's pseudo-fs may have multiple
file system instances on different servers associated with it. file system instances on different servers associated with it.
All such instances are considered replicas of one another. All such instances are considered replicas of one another.
o When a file system is present in a server's pseudo-fs, but there o When a file system is present in a server's pseudo-fs, but there
is no corresponding local file system, it is said to be "absent". is no corresponding local file system, it is said to be "absent".
In such cases, all associated instances will be accessed on other In such cases, all associated instances will be accessed on other
servers. servers.
Regarding terminology relating to attributes used in trunking Regarding terminology relating to attributes used in trunking
discovery and other multi-server namespace features: discovery and other multi-server namespace features:
o Location attributes include the fs_locations and fs_locations_info o File system location attributes include the fs_locations and
attributes. fs_locations_info attributes.
o Location entries are the individual file system locations in the o File system location entries provide the individual file system
location attributes. Each such entry specifies a server, in the locations within the file system location attributes. Each such
form of a host name or IP address, and an fs name, which entry specifies a server, in the form of a host name or IP
designates the location of the file system within the server's address, and an fs name, which designates the location of the file
pseudo-fs. A location entry designates a set of server endpoints system within the server's pseudo-fs. A file system location
to which the client may establish connections. There may be entry designates a set of server endpoints to which the client may
multiple endpoints because a host name may map to multiple network establish connections. There may be multiple endpoints because a
addresses and because multiple connection types may be used to host name may map to multiple network addresses and because
communicate with a single network address. However, all such multiple connection types may be used to communicate with a single
endpoints MUST provide a way of connecting to a single server. network address. However, all such endpoints MUST provide a way
The exact form of the location entry varies with the particular of connecting to a single server. The exact form of the location
location attribute used, as described in Section 4.3. entry varies with the particular file system location attribute
used, as described in Section 4.3.
o Location elements are derived from location entries and each o File system location elements are derived from location entries
describes a particular network access path, consisting of a and each describes a particular network access path, consisting of
network address and a location within the server's pseudo-fs. a network address and a location within the server's pseudo-fs.
Location elements need not appear within a location attribute, but Such location elements need not appear within a file system
the existence of each location element derives from a location attribute, but the existence of each location element
corresponding location entry. When a location entry specifies an derives from a corresponding location entry. When a location
IP address there is only a single corresponding location element. entry specifies an IP address there is only a single corresponding
Location entries that contain a host name, are resolved using DNS, location element. File system location entries that contain a
and may result in one or more location elements. All location host name, are resolved using DNS, and may result in one or more
elements consist of a location address which is the IP address of location elements. All location elements consist of a location
an interface to a server and an fs name which is the location of address which is the IP address of an interface to a server and an
the file system within the server's pseudo-fs. The fs name is fs name which is the location of the file system within the
empty if the server has no pseudo-fs and only a single exported server's pseudo-fs. The fs name is empty if the server has no
file system at the root filehandle. pseudo-fs and only a single exported file system at the root
filehandle.
o Two location elements are said to be server-trunkable if they o Two file system location elements are said to be server-trunkable
specify the same fs name and the location addresses are such that if they specify the same fs name and the location addresses are
the location addresses are server-trunkable. such that the location addresses are server-trunkable. When the
corresponding network paths are used, the client will always be
able to use client ID trunking, but will only be able to use
session trunking if the paths are also session-trunkable.
o Two location elements are said to be session-trunkable if they o Two file system location elements are said to be session-trunkable
specify the same fs name and the location addresses are such that if they specify the same fs name and the location addresses are
the location addresses are session-trunkable. such that the location addresses are session-trunkable. When the
corresponding network paths are used, the client will be able to
able to use either client ID trunking or session trunking.
Each set of server-trunkable location elements defines a set of Each set of server-trunkable location elements defines a set of
available network access paths to a particular file system. When available network access paths to a particular file system. When
there are multiple such file systems, each of which contains the same there are multiple such file systems, each of which contains the same
data, these file systems are considered replicas of one another. data, these file systems are considered replicas of one another.
Logically, such replication is symmetric, since the fs currently in Logically, such replication is symmetric, since the fs currently in
use and an alternate fs are replicas of each other. Often, in other use and an alternate fs are replicas of each other. Often, in other
documents, the term "replica" is not applied to the fs currently in documents, the term "replica" is not applied to the fs currently in
use, despite the fact that the replication relation is inherently use, despite the fact that the replication relation is inherently
symmetric. symmetric.
4.3. Location Attributes (as updated) 4.3. Updated Section 11.1 of [RFC5661] to be retitled "File System
Location Attributes"
NFSv4.1 contains RECOMMENDED attributes that provide information NFSv4.1 contains RECOMMENDED attributes that provide information
about how (i.e. at what network address and namespace position) a about how (i.e. at what network address and namespace position) a
given file system may be accessed. As a result, file systems in the given file system may be accessed. As a result, file systems in the
namespace of one server can be associated with one or more instances namespace of one server can be associated with one or more instances
of that file system on other servers. These attributes contain of that file system on other servers. These attributes contain file
location entries specifying a server address target (either as a DNS system location entries specifying a server address target (either as
name representing one or more IP addresses or as a specific IP a DNS name representing one or more IP addresses or as a specific IP
address) together with the pathname of that file system within the address) together with the pathname of that file system within the
associated single-server namespace. associated single-server namespace.
The fs_locations_info RECOMMENDED attribute allows specification of The fs_locations_info RECOMMENDED attribute allows specification of
one or more file system instance locations where the data one or more file system instance locations where the data
corresponding to a given file system may be found. This attribute corresponding to a given file system may be found. This attribute
provides to the client, in addition to specification of file system provides to the client, in addition to specification of file system
instance locations, other helpful information such as: instance locations, other helpful information such as:
o Information guiding choices among the various file system o Information guiding choices among the various file system
skipping to change at page 13, line 41 skipping to change at page 15, line 13
etc.). etc.).
o Information to help the client efficiently effect as seamless a o Information to help the client efficiently effect as seamless a
transition as possible among multiple file system instances, when transition as possible among multiple file system instances, when
and if that should be necessary. and if that should be necessary.
o Information helping to guide the selection of the appropriate o Information helping to guide the selection of the appropriate
connection type to be used when establishing a connection. connection type to be used when establishing a connection.
Within the fs_locations_info attribute, each fs_locations_server4 Within the fs_locations_info attribute, each fs_locations_server4
entry corresponds to a location entry with the fls_server field entry corresponds to a file system location entry with the fls_server
designating the server, with the location pathname within the field designating the server, with the location pathname within the
server's pseudo-fs given by the fl_rootpath field of the encompassing server's pseudo-fs given by the fl_rootpath field of the encompassing
fs_locations_item4. fs_locations_item4.
The fs_locations attribute defined in NFSv4.0 is also a part of The fs_locations attribute defined in NFSv4.0 is also a part of
NFSv4.1. This attribute only allows specification of the file system NFSv4.1. This attribute only allows specification of the file system
locations where the data corresponding to a given file system may be locations where the data corresponding to a given file system may be
found. Servers should make this attribute available whenever found. Servers should make this attribute available whenever
fs_locations_info is supported, but client use of fs_locations_info fs_locations_info is supported, but client use of fs_locations_info
is preferable, as it provides more information. is preferable, as it provides more information.
Within the fs_location attribute, each fs_location4 contains a Within the fs_location attribute, each fs_location4 contains a file
location entry with the server field designating the server and the system location entry with the server field designating the server
rootpath field giving the location pathname within the server's and the rootpath field giving the location pathname within the
pseudo-fs. server's pseudo-fs.
4.4. Re-organization of Sections 11.4 and 11.5 of RFC5661 4.4. Re-organization of Sections 11.4 and 11.5 of [RFC5661]
Previously, issues related to the fact that multiple location entries Previously, issues related to the fact that multiple location entries
directed the client to the same file system instance were dealt with directed the client to the same file system instance were dealt with
in a separate Section 11.5 of [RFC5661]. Because of the new in a separate Section 11.5 of [RFC5661]. Because of the new
treatment of trunking, these issues now belong within Section 4.5 treatment of trunking, these issues now belong within Section 4.5
below. below.
In this new section of the current document, trunking is dealt with In this new section of the current document, trunking is dealt with
in Section 4.5.2 together with the other uses of location information in Section 4.5.2 together with the other uses of file system location
described in Sections 4.5.4, 4.5.5, and 4.5.6. information described in Sections 4.5.4, 4.5.5, and 4.5.6.
4.5. Uses of Location Information (as updated) 4.5. Updated Section 11.4 of [RFC5661] to be retitled "Uses of File
System Location Information"
The location attributes (i.e. fs_locations and fs_locations_info), The file system location attributes (i.e. fs_locations and
together with the possibility of absent file systems, provide a fs_locations_info), together with the possibility of absent file
number of important facilities in providing reliable, manageable, and systems, provide a number of important facilities in providing
scalable data access. reliable, manageable, and scalable data access.
When a file system is present, these attributes can provide When a file system is present, these attributes can provide
o The locations of alternative replicas, to be used to access the o The locations of alternative replicas, to be used to access the
same data in the event of server failures, communications same data in the event of server failures, communications
problems, or other difficulties that make continued access to the problems, or other difficulties that make continued access to the
current replica impossible or otherwise impractical. Provision current replica impossible or otherwise impractical. Provision
and use of such alternate replicas is referred to as "replication" and use of such alternate replicas is referred to as "replication"
and is discussed in Section 4.5.4 below. and is discussed in Section 4.5.4 below.
o The network address(es) to be used to access the current file o The network address(es) to be used to access the current file
system instance or replicas of it. Client use of this information system instance or replicas of it. Client use of this information
is discussed in Section 4.5.2 below. is discussed in Section 4.5.2 below.
skipping to change at page 15, line 15 skipping to change at page 16, line 37
is discussed in Section 4.5.4 below. is discussed in Section 4.5.4 below.
Where a file system was previously absent, specification of file Where a file system was previously absent, specification of file
system location provides a means by which file systems located on one system location provides a means by which file systems located on one
server can be associated with a namespace defined by another server, server can be associated with a namespace defined by another server,
thus allowing a general multi-server namespace facility. A thus allowing a general multi-server namespace facility. A
designation of such a remote instance, in place of a file system designation of such a remote instance, in place of a file system
never previously present , is called a "pure referral" and is never previously present , is called a "pure referral" and is
discussed in Section 4.5.6 below. discussed in Section 4.5.6 below.
Because client support for location-related attributes is OPTIONAL, a Because client support for attributes related to file system location
server may (but is not required to) take action to hide migration and is OPTIONAL, a server may (but is not required to) take action to
referral events from such clients, by acting as a proxy, for example. hide migration and referral events from such clients, by acting as a
The server can determine the presence of client support from the proxy, for example. The server can determine the presence of client
arguments of the EXCHANGE_ID operation (see Section 14.3 in the support from the arguments of the EXCHANGE_ID operation (see
current document). Section 14.3 in the current document).
4.5.1. Combining Multiple Uses in a Single Attribute (to be added) 4.5.1. New section to be added as the first sub-section of Section 11.4
of [RFC5661] to be entitled "Combining Multiple Uses in a Single
Attribute"
A location attribute will sometimes contain information relating to A file system location attribute will sometimes contain information
the location of multiple replicas which may be used in different relating to the location of multiple replicas which may be used in
ways. different ways.
o Location entries that relate to the file system instance currently o File system location entries that relate to the file system
in use provide trunking information, allowing the client to find instance currently in use provide trunking information, allowing
additional network addresses by which the instance may be the client to find additional network addresses by which the
accessed. instance may be accessed.
o Location entries that provide information about replicas to which o File system location entries that provide information about
access is to be transferred. replicas to which access is to be transferred.
o Other location entries that relate to replicas that are available o Other file system location entries that relate to replicas that
to use in the event that access to the current replica becomes are available to use in the event that access to the current
unsatisfactory. replica becomes unsatisfactory.
In order to simplify client handling and allow the best choice of In order to simplify client handling and allow the best choice of
replicas to access, the server should adhere to the following replicas to access, the server should adhere to the following
guidelines. guidelines.
o All location entries that relate to a single file system instance o All file system location entries that relate to a single file
should be adjacent. system instance should be adjacent.
o Location entries that relate to the instance currently in use o File system location entries that relate to the instance currently
should appear first. in use should appear first.
o Location entries that relate to replica(s) to which migration is o File system location entries that relate to replica(s) to which
occurring should appear before replicas which are available for migration is occurring should appear before replicas which are
later use if the current replica should become inaccessible. available for later use if the current replica should become
inaccessible.
4.5.2. Location Attributes and Trunking (to be added) 4.5.2. New section to be added as the second sub-section of
Section 11.4 of [RFC5661] to be entitled "File System Location
Attributes and Trunking"
Trunking is the use of multiple connections between a client and Trunking is the use of multiple connections between a client and
server in order to increase the speed of data transfer. A client may server in order to increase the speed of data transfer. A client may
determine the set of network addresses to use to access a given file determine the set of network addresses to use to access a given file
system in a number of ways: system in a number of ways:
o When the name of the server is known to the client, it may use DNS o When the name of the server is known to the client, it may use DNS
to obtain a set of network addresses to use in accessing the to obtain a set of network addresses to use in accessing the
server. server.
o It may fetch the location attribute for the filesystem which will o It may fetch the file system location attribute for the filesystem
provide either the name of the server (which can be turned into a which will provide either the name of the server (which can be
set of network addresses using DNS), or it will find a set of turned into a set of network addresses using DNS), or it will find
server-trunkable location entries which can provide the addresses a set of server-trunkable location entries which can provide the
specified by the server as desirable to use to access the file addresses specified by the server as desirable to use to access
system in question. the file system in question.
The server can provide location entries that include either names or The server can provide location entries that include either names or
network addresses. It might use the latter form because of DNS- network addresses. It might use the latter form because of DNS-
related security concerns or because the set of addresses to be used related security concerns or because the set of addresses to be used
might require active management by the server. might require active management by the server.
Locations entries used to discover candidate addresses for use in Locations entries used to discover candidate addresses for use in
trunking are subject to change, as discussed in Section 4.5.7 below. trunking are subject to change, as discussed in Section 4.5.7 below.
The client may respond to such changes by using additional addresses The client may respond to such changes by using additional addresses
once they are verified or by ceasing to use existing ones. The once they are verified or by ceasing to use existing ones. The
server can force the client to cease using an address by returning server can force the client to cease using an address by returning
NFS4ERR_MOVED when that address is used to access a file system. NFS4ERR_MOVED when that address is used to access a file system.
This allows a transfer of client access which is similar to This allows a transfer of client access which is similar to
migration, although the same file system instance is accessed migration, although the same file system instance is accessed
throughout. throughout.
4.5.3. Location Attributes and Connection Type Selection (to be added) 4.5.3. New section to be added as the third sub-section of Section 11.4
of [RFC5661] to be entitled "File System Location Attributes and
Connection Type Selection"
Because of the need to support multiple connections, clients face the Because of the need to support multiple connections, clients face the
issue of determining the proper connection type to use when issue of determining the proper connection type to use when
establishing a connection to a given server network address. In some establishing a connection to a given server network address. In some
cases, this issue can be addressed through the use of the connection cases, this issue can be addressed through the use of the connection
"step-up" facility described in Section 18.16 of [RFC5661]. However, "step-up" facility described in Section 18.16 of [RFC5661]. However,
because there are cases is which that facility is not available, the because there are cases is which that facility is not available, the
client may have to choose a connection type with no possibility of client may have to choose a connection type with no possibility of
changing it within the scope of a single connection. changing it within the scope of a single connection.
The two location attributes differ as to the information made The two file system location attributes differ as to the information
available in this regard. Fs_locations provides no information to made available in this regard. Fs_locations provides no information
support connection type selection. As a result, clients supporting to support connection type selection. As a result, clients
multiple connection types would need to attempt to establish supporting multiple connection types would need to attempt to
connections using multiple connection types until the one preferred establish connections using multiple connection types until the one
by the client is successfully established. preferred by the client is successfully established.
Fs_locations_info provides a flag, FSLI4TF_RDMA flag. indicating Fs_locations_info provides a flag, FSLI4TF_RDMA flag. indicating
that RPC-over-RDMA support is available using the specified location that RPC-over-RDMA support is available using the specified location
entry. This flag makes it for a convenient for a client wishing to entry. This flag makes it for a convenient for a client wishing to
use RDMA, to establish a TCP connection and then convert to use of use RDMA, to establish a TCP connection and then convert to use of
RDMA. After establishing a TCP connection, the step-up facility, can RDMA. After establishing a TCP connection, the step-up facility, can
be used, if available, to convert that connection to RDMA mode. be used, if available, to convert that connection to RDMA mode.
Otherwise, if RDMA availability is indicated, a new RDMA connection Otherwise, if RDMA availability is indicated, a new RDMA connection
can be established and it can be bound to the session already can be established and it can be bound to the session already
established by the TCP connection, allowing the TCP connection to be established by the TCP connection, allowing the TCP connection to be
dropped and the session converted to further use in RDMA node. dropped and the session converted to further use in RDMA node.
4.5.4. File System Replication (as updated) 4.5.4. Updated Section 11.4.1 of [RFC5661] entitled "File System
Replication"
The fs_locations and fs_locations_info attributes provide alternative The fs_locations and fs_locations_info attributes provide alternative
locations, to be used to access data in place of or in addition to file system locations, to be used to access data in place of or in
the current file system instance. On first access to a file system, addition to the current file system instance. On first access to a
the client should obtain the set of alternate locations by file system, the client should obtain the set of alternate locations
interrogating the fs_locations or fs_locations_info attribute, with by interrogating the fs_locations or fs_locations_info attribute,
the latter being preferred. with the latter being preferred.
In the event that server failures, communications problems, or other In the event that server failures, communications problems, or other
difficulties make continued access to the current file system difficulties make continued access to the current file system
impossible or otherwise impractical, the client can use the alternate impossible or otherwise impractical, the client can use the alternate
locations as a way to get continued access to its data. locations as a way to get continued access to its data.
The alternate locations may be physical replicas of the (typically The alternate locations may be physical replicas of the (typically
read-only) file system data, or they may provide for the use of read-only) file system data, or they may provide for the use of
various forms of server clustering in which multiple servers provide various forms of server clustering in which multiple servers provide
alternate ways of accessing the same physical file system. How these alternate ways of accessing the same physical file system. How these
different modes of file system transition are represented within the different modes of file system transition are represented within the
fs_locations and fs_locations_info attributes and how the client fs_locations and fs_locations_info attributes and how the client
deals with file system transition issues will be discussed in detail deals with file system transition issues will be discussed in detail
below. below.
4.5.5. File System Migration (as updated) 4.5.5. Updated Section 11.4.2 of [RFC5661] entitled "File System
Migration"
When a file system is present and becomes absent, clients can be When a file system is present and becomes absent, clients can be
given the opportunity to have continued access to their data, at an given the opportunity to have continued access to their data, at an
alternate location, as specified by a location attribute. This alternate location, as specified by a file system location attribute.
migration of access to another replica includes the ability to retain This migration of access to another replica includes the ability to
locks across the transition, either by using lock reclaim or by retain locks across the transition, either by using lock reclaim or
taking advantage of Transparent State Migration. by taking advantage of Transparent State Migration.
Typically, a client will be accessing the file system in question, Typically, a client will be accessing the file system in question,
get an NFS4ERR_MOVED error, and then use a location attribute to get an NFS4ERR_MOVED error, and then use a file system location
determine the new location of the data. When fs_locations_info is attribute to determine the new location of the data. When
used, additional information will be available that will define the fs_locations_info is used, additional information will be available
nature of the client's handling of the transition to a new server. that will define the nature of the client's handling of the
transition to a new server.
Such migration can be helpful in providing load balancing or general Such migration can be helpful in providing load balancing or general
resource reallocation. The protocol does not specify how the file resource reallocation. The protocol does not specify how the file
system will be moved between servers. It is anticipated that a system will be moved between servers. It is anticipated that a
number of different server-to-server transfer mechanisms might be number of different server-to-server transfer mechanisms might be
used with the choice left to the server implementer. The NFSv4.1 used with the choice left to the server implementer. The NFSv4.1
protocol specifies the method used to communicate the migration event protocol specifies the method used to communicate the migration event
between client and server. between client and server.
The new location may be, in the case of various forms of server The new location may be, in the case of various forms of server
skipping to change at page 19, line 5 skipping to change at page 20, line 34
degree indicated by the fs_locations_info attribute). Where file degree indicated by the fs_locations_info attribute). Where file
systems are writable, a change made on the original file system must systems are writable, a change made on the original file system must
be visible on all migration targets. Where a file system is not be visible on all migration targets. Where a file system is not
writable but represents a read-only copy (possibly periodically writable but represents a read-only copy (possibly periodically
updated) of a writable file system, similar requirements apply to the updated) of a writable file system, similar requirements apply to the
propagation of updates. Any change visible in the original file propagation of updates. Any change visible in the original file
system must already be effected on all migration targets, to avoid system must already be effected on all migration targets, to avoid
any possibility that a client, in effecting a transition to the any possibility that a client, in effecting a transition to the
migration target, will see any reversion in file system state. migration target, will see any reversion in file system state.
4.5.6. Referrals (as updated) 4.5.6. Updated Section 11.4.3 of [RFC5661] entitled "Referrals"
Referrals allow the server to associate a file system namespace entry Referrals allow the server to associate a file system namespace entry
located on one server with a file system located on another server. located on one server with a file system located on another server.
When this includes the use of pure referrals, servers are provided a When this includes the use of pure referrals, servers are provided a
way of placing a file system in a location within the namespace way of placing a file system in a location within the namespace
essentially without respect to its physical location on a particular essentially without respect to its physical location on a particular
server. This allows a single server or a set of servers to present a server. This allows a single server or a set of servers to present a
multi-server namespace that encompasses file systems located on a multi-server namespace that encompasses file systems located on a
wider range of servers. Some likely uses of this facility include wider range of servers. Some likely uses of this facility include
establishment of site-wide or organization-wide namespaces, with the establishment of site-wide or organization-wide namespaces, with the
eventual possibility of combining such together into a truly global eventual possibility of combining such together into a truly global
namespace. namespace.
Referrals occur when a client determines, upon first referencing a Referrals occur when a client determines, upon first referencing a
position in the current namespace, that it is part of a new file position in the current namespace, that it is part of a new file
system and that the file system is absent. When this occurs, system and that the file system is absent. When this occurs,
typically upon receiving the error NFS4ERR_MOVED, the actual location typically upon receiving the error NFS4ERR_MOVED, the actual location
or locations of the file system can be determined by fetching the a or locations of the file system can be determined by fetching the a
locations attribute. attribute. locations attribute. attribute.
The locations attribute may designate a single file system location The file system location attribute may designate a single file system
or multiple file system locations, to be selected based on the needs location or multiple file system locations, to be selected based on
of the client. The server, in the fs_locations_info attribute, may the needs of the client. The server, in the fs_locations_info
specify priorities to be associated with various file system location attribute, may specify priorities to be associated with various file
choices. The server may assign different priorities to different system location choices. The server may assign different priorities
locations as reported to individual clients, in order to adapt to to different locations as reported to individual clients, in order to
client physical location or to effect load balancing. When both adapt to client physical location or to effect load balancing. When
read-only and read-write file systems are present, some of the read- both read-only and read-write file systems are present, some of the
only locations might not be absolutely up-to-date (as they would have read-only locations might not be absolutely up-to-date (as they would
to be in the case of replication and migration). Servers may also have to be in the case of replication and migration). Servers may
specify file system locations that include client-substituted also specify file system locations that include client-substituted
variables so that different clients are referred to different file variables so that different clients are referred to different file
systems (with different data contents) based on client attributes systems (with different data contents) based on client attributes
such as CPU architecture. such as CPU architecture.
When the fs_locations_info attribute is such that that there are When the fs_locations_info attribute is such that that there are
multiple possible targets listed, the relationships among them may be multiple possible targets listed, the relationships among them may be
important to the client in selecting which one to use. The same important to the client in selecting which one to use. The same
rules specified in Section 4.5.5 below regarding multiple migration rules specified in Section 4.5.5 below regarding multiple migration
targets apply to these multiple replicas as well. For example, the targets apply to these multiple replicas as well. For example, the
client might prefer a writable target on a server that has additional client might prefer a writable target on a server that has additional
skipping to change at page 20, line 25 skipping to change at page 22, line 7
namespace. The top-level referral file system or any segment may use namespace. The top-level referral file system or any segment may use
replicated referral file systems for higher availability. replicated referral file systems for higher availability.
Generally, multi-server namespaces are for the most part uniform, in Generally, multi-server namespaces are for the most part uniform, in
that the same data made available to one client at a given location that the same data made available to one client at a given location
in the namespace is made available to all clients at that location. in the namespace is made available to all clients at that location.
However, as described above, there are facilities provided that allow However, as described above, there are facilities provided that allow
different clients to be directed different sets of data, to enable different clients to be directed different sets of data, to enable
adaptation to such client characteristics as CPU architecture. adaptation to such client characteristics as CPU architecture.
4.5.7. Changes in a Location Attribute (to be added) 4.5.7. New section to be added after Section 11.4.3 of [RFC5661] to be
entitled "Changes in a File System Location Attribute"
Although clients will typically fetch a location attribute when first Although clients will typically fetch a file system location
accessing a file system and when NFS4ERR_MOVED is returned, a client attribute when first accessing a file system and when NFS4ERR_MOVED
can choose to fetch the attribute periodically, in which case the is returned, a client can choose to fetch the attribute periodically,
value fetched may change over time. in which case the value fetched may change over time.
For clients not prepared to access multiple replicas simultaneously For clients not prepared to access multiple replicas simultaneously
(see Section 8.1 of the current document), the handling of the (see Section 8.1 of the current document), the handling of the
various cases of change is as follows: various cases of change is as follows:
o Changes in the list of replicas or in the network addresses o Changes in the list of replicas or in the network addresses
associated with replicas do not require immediate action. The associated with replicas do not require immediate action. The
client will typically update its list of replicas to reflect the client will typically update its list of replicas to reflect the
new information. new information.
skipping to change at page 21, line 23 skipping to change at page 23, line 5
with one currently in use, the client is free to use the new with one currently in use, the client is free to use the new
replica immediately. replica immediately.
o When a replica currently in use is deleted from the list, the o When a replica currently in use is deleted from the list, the
client need not cease using it immediately. However, since the client need not cease using it immediately. However, since the
server may subsequently force such use to cease (by returning server may subsequently force such use to cease (by returning
NFS4ERR_MOVED), clients might decide to limit the need for later NFS4ERR_MOVED), clients might decide to limit the need for later
state transfer. For example, new opens might be done on other state transfer. For example, new opens might be done on other
replicas, rather than on one not present in the list. replicas, rather than on one not present in the list.
5. Re-organization of Section 11.7 of RFC5661 5. Re-organization of Section 11.7 of [RFC5661]
The material in Section 11.7 of [RFC5661] has been reorganized and The material in Section 11.7 of [RFC5661] has been reorganized and
augmented as specified below: augmented as specified below:
o Because there can be a shift of the network access paths used to o Because there can be a shift of the network access paths used to
access a file system instance without any shift between replicas, access a file system instance without any shift between replicas,
a new Section 6 in the current document distinguishes between a new Section 6 in the current document distinguishes between
those cases in which there is a shift between distinct replicas those cases in which there is a shift between distinct replicas
and those involving a shift in network access paths with no shift and those involving a shift in network access paths with no shift
between replicas. between replicas.
skipping to change at page 22, line 9 skipping to change at page 23, line 37
o The additional Section 10 in the current document discusses the o The additional Section 10 in the current document discusses the
client's response to access transitions and how it determines client's response to access transitions and how it determines
whether migration has occurred, and how it gets access to any whether migration has occurred, and how it gets access to any
transferred locking and session state. transferred locking and session state.
o The additional Section 11 in the current document discusses the o The additional Section 11 in the current document discusses the
responsibilities of the source and destination servers when responsibilities of the source and destination servers when
transferring locking and session state. transferring locking and session state.
6. Overview of File Access Transitions (to be added) 6. New section to be added after Section 11.6 of [RFC5661] to be
entitled "Overview of File Access Transitions"
File access transitions are of two types: File access transitions are of two types:
o Those that involve a transition from accessing the current replica o Those that involve a transition from accessing the current replica
to another one in connection with either replication or migration. to another one in connection with either replication or migration.
How these are dealt with is discussed in Section 8 of the current How these are dealt with is discussed in Section 8 of the current
document. document.
o Those in which access to the current file system instance is o Those in which access to the current file system instance is
retained, while the network path used to access that instance is retained, while the network path used to access that instance is
changed. This case is discussed in Section 7 of the current changed. This case is discussed in Section 7 of the current
document. document.
7. Effecting Network Endpoint Transitions (to be added) 7. New section to be added second after Section 11.6 of [RFC5661] to be
entitled "Effecting Network Endpoint Transitions"
The endpoints used to access a particular file system instance may The endpoints used to access a particular file system instance may
change in a number of ways, as listed below. In each of these cases, change in a number of ways, as listed below. In each of these cases,
the same filehandles, stateids, client IDs and session are used to the same filehandles, stateids, client IDs and session are used to
continue access, with a continuity of lock state. continue access, with a continuity of lock state.
o When use of a particular address is to cease and there is also one o When use of a particular address is to cease and there is also one
currently in use which is server-trunkable with it, requests that currently in use which is server-trunkable with it, requests that
would have been issued on the address whose use is to be would have been issued on the address whose use is to be
discontinued can be issued on the remaining address(es). When an discontinued can be issued on the remaining address(es). When an
address is not a session-trunkable one, the request might need to address is not a session-trunkable one, the request might need to
be modified to reflect the fact that a different session will be be modified to reflect the fact that a different session will be
used. used.
o When use of a particular connection is to cease, as indicated by o When use of a particular connection is to cease, as indicated by
receiving NFS4ERR_MOVED when using that connection but that receiving NFS4ERR_MOVED when using that connection but that
address is still indicated as accessible according to the address is still indicated as accessible according to the
appropriate location entries, it is likely that requests can be appropriate file system location entries, it is likely that
issued on a new connection of a different connection type, once requests can be issued on a new connection of a different
that connection is established. Since any two server endpoints connection type, once that connection is established. Since any
that share a network address are inherently session-trunkable, the two server endpoints that share a network address are inherently
client can use BIND_CONN_TO_SESSION to access the existing session session-trunkable, the client can use BIND_CONN_TO_SESSION to
using the new connection and proceed to access the file system access the existing session using the new connection and proceed
using the new connection. to access the file system using the new connection.
o When there are no potential replacement addresses in use but there o When there are no potential replacement addresses in use but there
are valid addresses session-trunkable with the one whose use is to are valid addresses session-trunkable with the one whose use is to
be discontinued, the client can use BIND_CONN_TO_SESSION to access be discontinued, the client can use BIND_CONN_TO_SESSION to access
the existing session using the new address. Although the target the existing session using the new address. Although the target
session will generally be accessible, there may be cases in which session will generally be accessible, there may be cases in which
that session in no longer accessible, in which case a new session that session in no longer accessible, in which case a new session
can be created to provide the client continued access to the can be created to provide the client continued access to the
existing instance. existing instance.
skipping to change at page 23, line 21 skipping to change at page 25, line 5
to be discontinued, other server-trunkable addresses may be used to be discontinued, other server-trunkable addresses may be used
to provide continued access. Although use of CREATE_SESSION is to provide continued access. Although use of CREATE_SESSION is
available to provide continued access to the existing instance, available to provide continued access to the existing instance,
servers have the option of providing continued access to the servers have the option of providing continued access to the
existing session through the new network access path in a fashion existing session through the new network access path in a fashion
similar to that provided by session migration (see Section 9 of similar to that provided by session migration (see Section 9 of
the current document). To take advantage of this possibility, the current document). To take advantage of this possibility,
clients can perform an initial BIND_CONN_TO_SESSION, as in the clients can perform an initial BIND_CONN_TO_SESSION, as in the
previous case, and use CREATE_SESSION only if that fails. previous case, and use CREATE_SESSION only if that fails.
8. Effecting File System Transitions (as updated) 8. Updated Section 11.7 of [RFC5661] entitled "Effecting File System
Transitions"
There are a range of situations in which there is a change to be There are a range of situations in which there is a change to be
effected in the set of replicas used to access a particular file effected in the set of replicas used to access a particular file
system. Some of these may involve an expansion or contraction of the system. Some of these may involve an expansion or contraction of the
set of replicas used as discussed in Section 8.1 below. set of replicas used as discussed in Section 8.1 below.
For reasons explained in that section, most transitions will involve For reasons explained in that section, most transitions will involve
a transition from a single replica to a corresponding replacement a transition from a single replica to a corresponding replacement
replica. When effecting replica transition, some types of sharing replica. When effecting replica transition, some types of sharing
between the replicas may affect handling of the transition as between the replicas may affect handling of the transition as
skipping to change at page 24, line 14 skipping to change at page 25, line 45
o The client handling of transitions, including determining how to o The client handling of transitions, including determining how to
deal with the various means that the server might take to supply deal with the various means that the server might take to supply
effective continuity of locking state are discussed in Section 10 effective continuity of locking state are discussed in Section 10
of the current document. of the current document.
o The servers' (source and destination) responsibilities in o The servers' (source and destination) responsibilities in
effecting Transparent Migration of locking and session state are effecting Transparent Migration of locking and session state are
discussed in Section 11 of the current document. discussed in Section 11 of the current document.
8.1. File System Transitions and Simultaneous Access (as updated) 8.1. Updated Section 11.7.1 of [RFC5661] entitled "File System
Transitions and Simultaneous Access"
The fs_locations_info attribute (described in Section 11.10.1 of The fs_locations_info attribute (described in Section 11.10.1 of
[RFC5661] and Section 12.2 of this document) may indicate that two [RFC5661] and Section 12.2 of this document) may indicate that two
replicas may be used simultaneously (see Section 11.7.2.1 of replicas may be used simultaneously (see Section 11.7.2.1 of
[RFC5661] for details). Although situations in which multiple [RFC5661] for details). Although situations in which multiple
replicas may be accessed simultaneously are somewhat similar to those replicas may be accessed simultaneously are somewhat similar to those
in which a single replica is accessed by multiple network addresses, in which a single replica is accessed by multiple network addresses,
there are important differences, since locking state is not shared there are important differences, since locking state is not shared
among multiple replicas. among multiple replicas.
skipping to change at page 24, line 47 skipping to change at page 26, line 30
For example, if one of the replicas become unavailable, access will For example, if one of the replicas become unavailable, access will
be transferred to a different replica, also capable of simultaneous be transferred to a different replica, also capable of simultaneous
access with the one still in use. access with the one still in use.
When there is no such replica, the transition may be to the replica When there is no such replica, the transition may be to the replica
already in use. At this point, the client has a choice between already in use. At this point, the client has a choice between
merging the locking state for the two replicas under the aegis of the merging the locking state for the two replicas under the aegis of the
sole replica in use or treating these separately, until another sole replica in use or treating these separately, until another
replica capable of simultaneous access presents itself. replica capable of simultaneous access presents itself.
8.2. Filehandles and File System Transitions (as updated) 8.2. Updated Section 11.7.3 of [RFC5661] entitled "Filehandles and File
System Transitions"
There are a number of ways in which filehandles can be handled across There are a number of ways in which filehandles can be handled across
a file system transition. These can be divided into two broad a file system transition. These can be divided into two broad
classes depending upon whether the two file systems across which the classes depending upon whether the two file systems across which the
transition happens share sufficient state to effect some sort of transition happens share sufficient state to effect some sort of
continuity of file system handling. continuity of file system handling.
When there is no such cooperation in filehandle assignment, the two When there is no such cooperation in filehandle assignment, the two
file systems are reported as being in different handle classes. In file systems are reported as being in different handle classes. In
this case, all filehandles are assumed to expire as part of the file this case, all filehandles are assumed to expire as part of the file
skipping to change at page 25, line 22 skipping to change at page 27, line 7
FH4_VOL_MIGRATION bit, which only affects behavior when FH4_VOL_MIGRATION bit, which only affects behavior when
fs_locations_info is not available. fs_locations_info is not available.
When there is cooperation in filehandle assignment, the two file When there is cooperation in filehandle assignment, the two file
systems are reported as being in the same handle classes. In this systems are reported as being in the same handle classes. In this
case, persistent filehandles remain valid after the file system case, persistent filehandles remain valid after the file system
transition, while volatile filehandles (excluding those that are only transition, while volatile filehandles (excluding those that are only
volatile due to the FH4_VOL_MIGRATION bit) are subject to expiration volatile due to the FH4_VOL_MIGRATION bit) are subject to expiration
on the target server. on the target server.
8.3. Fileids and File System Transitions (as updated) 8.3. Updated Section 11.7.4 of [RFC5661] entitled "Fileids and File
System Transitions"
In NFSv4.0, the issue of continuity of fileids in the event of a file In NFSv4.0, the issue of continuity of fileids in the event of a file
system transition was not addressed. The general expectation had system transition was not addressed. The general expectation had
been that in situations in which the two file system instances are been that in situations in which the two file system instances are
created by a single vendor using some sort of file system image copy, created by a single vendor using some sort of file system image copy,
fileids would be consistent across the transition, while in the fileids would be consistent across the transition, while in the
analogous multi-vendor transitions they would not. This poses analogous multi-vendor transitions they would not. This poses
difficulties, especially for the client without special knowledge of difficulties, especially for the client without special knowledge of
the transition mechanisms adopted by the server. Note that although the transition mechanisms adopted by the server. Note that although
fileid is not a REQUIRED attribute, many servers support fileids and fileid is not a REQUIRED attribute, many servers support fileids and
skipping to change at page 26, line 31 skipping to change at page 28, line 18
are no reliable filehandles across a transition event (either because are no reliable filehandles across a transition event (either because
there is no filehandle continuity or because the filehandles are there is no filehandle continuity or because the filehandles are
volatile), the client is in a position where it cannot verify that volatile), the client is in a position where it cannot verify that
files it was accessing before the transition are the same objects. files it was accessing before the transition are the same objects.
It is forced to assume that no object has been renamed, and, unless It is forced to assume that no object has been renamed, and, unless
there are guarantees that provide this (e.g., the file system is there are guarantees that provide this (e.g., the file system is
read-only), problems for applications may occur. Therefore, use of read-only), problems for applications may occur. Therefore, use of
such configurations should be limited to situations where the such configurations should be limited to situations where the
problems that this may cause can be tolerated. problems that this may cause can be tolerated.
8.4. Fsids and File System Transitions (as updated) 8.4. Updated section 11.7.5 of [RFC5661] entitled "Fsids and File
System Transitions"
Since fsids are generally only unique on a per-server basis, it is Since fsids are generally only unique on a per-server basis, it is
likely that they will change during a file system transition. likely that they will change during a file system transition.
Clients should not make the fsids received from the server visible to Clients should not make the fsids received from the server visible to
applications since they may not be globally unique, and because they applications since they may not be globally unique, and because they
may change during a file system transition event. Applications are may change during a file system transition event. Applications are
best served if they are isolated from such transitions to the extent best served if they are isolated from such transitions to the extent
possible. possible.
Although normally a single source file system will transition to a Although normally a single source file system will transition to a
single target file system, there is a provision for splitting a single target file system, there is a provision for splitting a
single source file system into multiple target file systems, by single source file system into multiple target file systems, by
specifying the FSLI4F_MULTI_FS flag. specifying the FSLI4F_MULTI_FS flag.
8.4.1. File System Splitting (as updated) 8.4.1. Updated section 11.7.5.1 of [RFC5661] entitled "File System
Splitting"
When a file system transition is made and the fs_locations_info When a file system transition is made and the fs_locations_info
indicates that the file system in question might be split into indicates that the file system in question might be split into
multiple file systems (via the FSLI4F_MULTI_FS flag), the client multiple file systems (via the FSLI4F_MULTI_FS flag), the client
SHOULD do GETATTRs to determine the fsid attribute on all known SHOULD do GETATTRs to determine the fsid attribute on all known
objects within the file system undergoing transition to determine the objects within the file system undergoing transition to determine the
new file system boundaries. new file system boundaries.
Clients might choose to maintain the fsids passed to existing Clients might choose to maintain the fsids passed to existing
applications by mapping all of the fsids for the descendant file applications by mapping all of the fsids for the descendant file
systems to the common fsid used for the original file system. systems to the common fsid used for the original file system.
Splitting a file system can be done on a transition between file Splitting a file system can be done on a transition between file
systems of the same fileid class, since the fact that fileids are systems of the same fileid class, since the fact that fileids are
unique within the source file system ensure they will be unique in unique within the source file system ensure they will be unique in
each of the target file systems. each of the target file systems.
8.5. The Change Attribute and File System Transitions (as updated) 8.5. Updated Section 11.7.6 of [RFC5661] entitled "The Change Attribute
and File System Transitions"
Since the change attribute is defined as a server-specific one, Since the change attribute is defined as a server-specific one,
change attributes fetched from one server are normally presumed to be change attributes fetched from one server are normally presumed to be
invalid on another server. Such a presumption is troublesome since invalid on another server. Such a presumption is troublesome since
it would invalidate all cached change attributes, requiring it would invalidate all cached change attributes, requiring
refetching. Even more disruptive, the absence of any assured refetching. Even more disruptive, the absence of any assured
continuity for the change attribute means that even if the same value continuity for the change attribute means that even if the same value
is retrieved on refetch, no conclusions can be drawn as to whether is retrieved on refetch, no conclusions can be drawn as to whether
the object in question has changed. The identical change attribute the object in question has changed. The identical change attribute
could be merely an artifact of a modified file with a different could be merely an artifact of a modified file with a different
change attribute construction algorithm, with that new algorithm just change attribute construction algorithm, with that new algorithm just
happening to result in an identical change value. happening to result in an identical change value.
When the two file systems have consistent change attribute formats, When the two file systems have consistent change attribute formats,
and this fact is communicated to the client by reporting in the same and this fact is communicated to the client by reporting in the same
change class, the client may assume a continuity of change attribute change class, the client may assume a continuity of change attribute
construction and handle this situation just as it would be handled construction and handle this situation just as it would be handled
without any file system transition. without any file system transition.
8.6. Write Verifiers and File System Transitions (as updated) 8.6. Updated Section 11.7.8 of [RFC5661] entitled "Write Verifiers and
File System Transitions"
In a file system transition, the two file systems might be clustered In a file system transition, the two file systems might be clustered
in the handling of unstably written data. When this is the case, and in the handling of unstably written data. When this is the case, and
the two file systems belong to the same write-verifier class, write the two file systems belong to the same write-verifier class, write
verifiers returned from one system may be compared to those returned verifiers returned from one system may be compared to those returned
by the other and superfluous writes avoided. by the other and superfluous writes avoided.
When two file systems belong to different write-verifier classes, any When two file systems belong to different write-verifier classes, any
verifier generated by one must not be compared to one provided by the verifier generated by one must not be compared to one provided by the
other. Instead, the two verifiers should be treated as not equal other. Instead, the two verifiers should be treated as not equal
even when the values are identical. even when the values are identical.
8.7. Readdir Cookies and Verifiers and File System Transitions (as 8.7. Updated Section 11.7.9 of [RFC5661] entitled "Readdir Cookies and
updated) Verifiers and File System Transitions)"
In a file system transition, the two file systems might be consistent In a file system transition, the two file systems might be consistent
in their handling of READDIR cookies and verifiers. When this is the in their handling of READDIR cookies and verifiers. When this is the
case, and the two file systems belong to the same readdir class, case, and the two file systems belong to the same readdir class,
READDIR cookies and verifiers from one system may be recognized by READDIR cookies and verifiers from one system may be recognized by
the other and READDIR operations started on one server may be validly the other and READDIR operations started on one server may be validly
continued on the other, simply by presenting the cookie and verifier continued on the other, simply by presenting the cookie and verifier
returned by a READDIR operation done on the first file system to the returned by a READDIR operation done on the first file system to the
second. second.
When two file systems belong to different readdir classes, any When two file systems belong to different readdir classes, any
READDIR cookie and verifier generated by one is not valid on the READDIR cookie and verifier generated by one is not valid on the
second, and must not be presented to that server by the client. The second, and must not be presented to that server by the client. The
client should act as if the verifier was rejected. client should act as if the verifier was rejected.
8.8. File System Data and File System Transitions (as updated) 8.8. Updated Section 11.7.10 of [RFC5661] entitled "File System Data
and File System Transitions"
When multiple replicas exist and are used simultaneously or in When multiple replicas exist and are used simultaneously or in
succession by a client, applications using them will normally expect succession by a client, applications using them will normally expect
that they contain either the same data or data that is consistent that they contain either the same data or data that is consistent
with the normal sorts of changes that are made by other clients with the normal sorts of changes that are made by other clients
updating the data of the file system (with metadata being the same to updating the data of the file system (with metadata being the same to
the degree indicated by the fs_locations_info attribute). However, the degree indicated by the fs_locations_info attribute). However,
when multiple file systems are presented as replicas of one another, when multiple file systems are presented as replicas of one another,
the precise relationship between the data of one and the data of the precise relationship between the data of one and the data of
another is not, as a general matter, specified by the NFSv4.1 another is not, as a general matter, specified by the NFSv4.1
skipping to change at page 29, line 38 skipping to change at page 31, line 29
transition to a replica, will see any reversion in file system transition to a replica, will see any reversion in file system
state. The specific means of this guarantee varies based on the state. The specific means of this guarantee varies based on the
value of the fss_type field that is reported as part of the value of the fss_type field that is reported as part of the
fs_status attribute (see Section 11.11 of [RFC5661]). Since these fs_status attribute (see Section 11.11 of [RFC5661]). Since these
file systems are presumed to be unsuitable for simultaneous use, file systems are presumed to be unsuitable for simultaneous use,
there is no specification of how locking is handled; in general, there is no specification of how locking is handled; in general,
locks obtained on one file system will be separate from those on locks obtained on one file system will be separate from those on
others. Since these are expected to be read-only file systems, others. Since these are expected to be read-only file systems,
this is not likely to pose an issue for clients or applications. this is not likely to pose an issue for clients or applications.
8.9. Lock State and File System Transitions (as updated) 8.9. Updated Section 11.7.7 of [RFC5661] entitled "Lock State and File
System Transitions"
While accessing a file system, clients obtain locks enforced by the While accessing a file system, clients obtain locks enforced by the
server which may prevent actions by other clients that are server which may prevent actions by other clients that are
inconsistent with those locks. inconsistent with those locks.
When access is transferred between replicas, clients need to be When access is transferred between replicas, clients need to be
assured that the actions disallowed by holding these locks cannot assured that the actions disallowed by holding these locks cannot
have occurred during the transition. This can be ensured by the have occurred during the transition. This can be ensured by the
methods below. Unless at least one of these is implemented, clients methods below. Unless at least one of these is implemented, clients
will not be assured of continuity of lock possession across a will not be assured of continuity of lock possession across a
skipping to change at page 30, line 35 skipping to change at page 32, line 25
State Migration is the only available means of providing the needed State Migration is the only available means of providing the needed
functionality. functionality.
It should be noted that these two methods are not mutually exclusive It should be noted that these two methods are not mutually exclusive
and that a server might well provide both. In particular, if there and that a server might well provide both. In particular, if there
is some circumstance preventing a specific lock from being is some circumstance preventing a specific lock from being
transferred transparently, the destination server can allow it to be transferred transparently, the destination server can allow it to be
reclaimed, by implementing a per-fs grace period for the migrated reclaimed, by implementing a per-fs grace period for the migrated
file system. file system.
9. Transferring State upon Migration (to be added) 9. New section to be added after Section 11.11 of [RFC5661] to be
entitled "Transferring State upon Migration"
When the transition is a result of a server-initiated decision to When the transition is a result of a server-initiated decision to
transition access and the source and destination servers have transition access and the source and destination servers have
implemented appropriate co-operation, it is possible to: implemented appropriate co-operation, it is possible to:
o Transfer locking state from the source to the destination server, o Transfer locking state from the source to the destination server,
in a fashion similar to that provided by Transparent State in a fashion similar to that provided by Transparent State
Migration in NFSv4.0, as described in [RFC7931]. Server Migration in NFSv4.0, as described in [RFC7931]. Server
responsibilities are described in Section 11.2 of the current responsibilities are described in Section 11.2 of the current
document. document.
o Transfer session state from the source to the destination server. o Transfer session state from the source to the destination server.
Server responsibilities in effecting such a transfer are described Server responsibilities in effecting such a transfer are described
in Section 11.3 of the current document. in Section 11.3 of the current document.
The means by which the client determines which of these transfer The means by which the client determines which of these transfer
events has occurred are described in Section 10 of the current events has occurred are described in Section 10 of the current
document. document.
9.1. Transparent State Migration and pNFS (to be added) 9.1. Only sub-section within new section to be added to [RFC5661] to be
entitled "Transparent State Migration and pNFS"
When pNFS is involved, the protocol is capable of supporting: When pNFS is involved, the protocol is capable of supporting:
o Migration of the Metadata Server (MDS), leaving the Data Servers o Migration of the Metadata Server (MDS), leaving the Data Servers
(DS's) in place. (DS's) in place.
o Migration of the file system as a whole, including the MDS and o Migration of the file system as a whole, including the MDS and
associated DS's. associated DS's.
o Replacement of one DS by another. o Replacement of one DS by another.
o Migration of a pNFS file system to one in which pNFS is not used. o Migration of a pNFS file system to one in which pNFS is not used.
o Migration of a file system not using pNFS to one in which layouts o Migration of a file system not using pNFS to one in which layouts
are available. are available.
Note that migration per se is only involved in the transfer of the
MDS function. Although the servicing of a layout may be transferred
from one data server to another, this not done using the file system
location attributes. The MDS can effect such transfers by recalling/
revoking existing layouts and granting new ones on a different data
server.
Migration of the MDS function is directly supported by Transparent Migration of the MDS function is directly supported by Transparent
State Migration. Layout state will normally be transparently State Migration. Layout state will normally be transparently
transferred, just as other state is. As a result, Transparent State transferred, just as other state is. As a result, Transparent State
Migration provides a framework in which, given appropriate inter-MDS Migration provides a framework in which, given appropriate inter-MDS
data transfer, one MDS can be substituted for another. data transfer, one MDS can be substituted for another.
Migration of the file system function as a whole can be accomplished Migration of the file system function as a whole can be accomplished
by recalling all layouts as part of the initial phase of the by recalling all layouts as part of the initial phase of the
migration process. As a result, IO will be done through the MDS migration process. As a result, IO will be done through the MDS
during the migration process, and new layouts can be granted once the during the migration process, and new layouts can be granted once the
skipping to change at page 32, line 9 skipping to change at page 34, line 5
such but can be effected by an MDS recalling layouts for the DS to be such but can be effected by an MDS recalling layouts for the DS to be
replaced and issuing new ones to be served by the successor DS. replaced and issuing new ones to be served by the successor DS.
Migration may transfer a file system from a server which does not Migration may transfer a file system from a server which does not
support pNFS to one which does. In order to properly adapt to this support pNFS to one which does. In order to properly adapt to this
situation, clients which support pNFS, but function adequately in its situation, clients which support pNFS, but function adequately in its
absence should check for pNFS support when a file system is migrated absence should check for pNFS support when a file system is migrated
and be prepared to use pNFS when support is available on the and be prepared to use pNFS when support is available on the
destination. destination.
10. Client Responsibilities when Access is Transitioned (to be added) 10. New section to be added second after Section 11.11 of [RFC5661] to
be entitled "Client Responsibilities when Access is Transitioned"
For a client to respond to an access transition, it must become aware For a client to respond to an access transition, it must become aware
of it. The ways in which this can happen are discussed in of it. The ways in which this can happen are discussed in
Section 10.1 which discusses indications that a specific file system Section 10.1 which discusses indications that a specific file system
access path has transitioned as well as situations in which access path has transitioned as well as situations in which
additional activity is necessary to determine the set of file systems additional activity is necessary to determine the set of file systems
that have been migrated. Section 10.2 goes on to complete the that have been migrated. Section 10.2 goes on to complete the
discussion of how the set of migrated file systems might be discussion of how the set of migrated file systems might be
determined. Sections 10.3 through 10.5 discuss how the client should determined. Sections 10.3 through 10.5 discuss how the client should
deal with each transition it becomes aware of, either directly or as deal with each transition it becomes aware of, either directly or as
skipping to change at page 32, line 37 skipping to change at page 34, line 34
o "Migration recovery" to that subset of transition recovery which o "Migration recovery" to that subset of transition recovery which
applies when the file system has migrated to a different replica. applies when the file system has migrated to a different replica.
o "Migration discovery" refers to the process of determining which o "Migration discovery" refers to the process of determining which
file system(s) have been migrated. It is necessary to avoid a file system(s) have been migrated. It is necessary to avoid a
situation in which leases could expire when a file system is not situation in which leases could expire when a file system is not
accessed for a long period of time, since a client unaware of the accessed for a long period of time, since a client unaware of the
migration might be referencing an unmigrated file system and not migration might be referencing an unmigrated file system and not
renewing the lease associated with the migrated file system. renewing the lease associated with the migrated file system.
10.1. Client Transition Notifications (to be added) 10.1. First sub-section within new section to be added to [RFC5661] to
be entitled "Client Transition Notifications"
When there is a change in the network access path which a client is When there is a change in the network access path which a client is
to use to access a file system, there are a number of related status to use to access a file system, there are a number of related status
indications with which clients need to deal: indications with which clients need to deal:
o If an attempt is made to use or return a filehandle within a file o If an attempt is made to use or return a filehandle within a file
system that is no longer accessible at the address previously used system that is no longer accessible at the address previously used
to access it, the error NFS4ERR_MOVED is returned. to access it, the error NFS4ERR_MOVED is returned.
Exceptions are made to allow such file handles to be used when Exceptions are made to allow such file handles to be used when
interrogating a location attribute. This enables a client to interrogating a file system location attribute. This enables a
determine a new replica's location or a new network access path. client to determine a new replica's location or a new network
access path.
This condition continues on subsequent attempts to access the file This condition continues on subsequent attempts to access the file
system in question. The only way the client can avoid the error system in question. The only way the client can avoid the error
is to cease accessing the filesystem in question at its old server is to cease accessing the filesystem in question at its old server
location and access it instead using a different address at which location and access it instead using a different address at which
it is now available. it is now available.
o Whenever a SEQUENCE operation is sent by a client to a server o Whenever a SEQUENCE operation is sent by a client to a server
which generated state held on that client which is associated with which generated state held on that client which is associated with
a file system that is no longer accessible on the server at which a file system that is no longer accessible on the server at which
it was previously available, a lease-migrated indication, in the it was previously available, a lease-migrated indication, in the
form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in form the SEQ4_STATUS_LEASE_MOVED status bit being set, appears in
the response. the response.
This condition continues until the client acknowledges the This condition continues until the client acknowledges the
notification by fetching a location attribute for the file system notification by fetching a file system location attribute for the
whose network access path is being changed. When there are file system whose network access path is being changed. When
multiple such file systems, a location attribute for each such there are multiple such file systems, a location attribute for
file system needs to be fetched. The location attribute for all each such file system needs to be fetched. The location attribute
migrated file system needs to be fetched in order to clear the for all migrated file system needs to be fetched in order to clear
condition. Even after the condition is cleared, the client needs the condition. Even after the condition is cleared, the client
to respond by using the location information to access the file needs to respond by using the location information to access the
system at its new location to ensure that leases are not file system at its new location to ensure that leases are not
needlessly expired. needlessly expired.
Unlike the case of NFSv4.0, in which the corresponding conditions are Unlike the case of NFSv4.0, in which the corresponding conditions are
both errors and thus mutually exclusive, in NFSv4.1 the client can, both errors and thus mutually exclusive, in NFSv4.1 the client can,
and often will, receive both indications on the same request. As a and often will, receive both indications on the same request. As a
result, implementations need to address the question of how to co- result, implementations need to address the question of how to co-
ordinate the necessary recovery actions when both indications arrive ordinate the necessary recovery actions when both indications arrive
in the response to the same request. It should be noted that when in the response to the same request. It should be noted that when
processing an NFSv4 COMPOUND, the server will normally decide whether processing an NFSv4 COMPOUND, the server will normally decide whether
SEQ4_STATUS_LEASE_MOVED is to be set before it determines which file SEQ4_STATUS_LEASE_MOVED is to be set before it determines which file
skipping to change at page 35, line 4 skipping to change at page 36, line 49
the fact, discussed above, that the two indications are not mutually the fact, discussed above, that the two indications are not mutually
exclusive, there are number of issues that are important in exclusive, there are number of issues that are important in
considering implementation of migration discovery, as discussed in considering implementation of migration discovery, as discussed in
Section 10.2. Section 10.2.
Because of the absence of NFSV4ERR_LEASE_MOVED, it is possible for Because of the absence of NFSV4ERR_LEASE_MOVED, it is possible for
file systems whose access path has not changed to be successfully file systems whose access path has not changed to be successfully
accessed on a given server even though recovery is necessary for accessed on a given server even though recovery is necessary for
other file systems on the same server. As a result, access can go on other file systems on the same server. As a result, access can go on
while, while,
o The migration discovery process is going on for that server. o The migration discovery process is going on for that server.
o The transition recovery process is going on for on other file o The transition recovery process is going on for on other file
systems connected to that server. systems connected to that server.
10.2. Performing Migration Discovery (to be added) 10.2. Second sub-section within new section to be added to [RFC5661] to
be entitled "Performing Migration Discovery"
Migration discovery can be performed in the same context as Migration discovery can be performed in the same context as
transition recovery, allowing recovery for each migrated file system transition recovery, allowing recovery for each migrated file system
to be invoked as it is discovered. Alternatively, it may be done in to be invoked as it is discovered. Alternatively, it may be done in
a separate migration discovery thread, allowing migration discovery a separate migration discovery thread, allowing migration discovery
to be done in parallel with one or more instances of transition to be done in parallel with one or more instances of transition
recovery. recovery.
In either case, because the lease-migrated indication does not result In either case, because the lease-migrated indication does not result
in an error. other access to file systems on the server can proceed in an error. other access to file systems on the server can proceed
skipping to change at page 36, line 24 skipping to change at page 38, line 24
o completion/verification of migration discovery processing, in o completion/verification of migration discovery processing, in
which the possible completion of migration discovery processing which the possible completion of migration discovery processing
needs to be verified. needs to be verified.
Given that framework, migration discovery processing would proceed as Given that framework, migration discovery processing would proceed as
follows. follows.
o While in the normal-operation state, the thread performing o While in the normal-operation state, the thread performing
discovery would fetch, for successive file systems known to the discovery would fetch, for successive file systems known to the
client on the server being worked on, a location attribute plus client on the server being worked on, a file system location
the fs_status attribute. attribute plus the fs_status attribute.
o If the fs_status attribute indicates that the file system is a o If the fs_status attribute indicates that the file system is a
migrated one (i.e. fss_absent is true and fss_type != migrated one (i.e. fss_absent is true and fss_type !=
STATUS4_REFERRAL) and thus that it is likely that the fetch of the STATUS4_REFERRAL) and thus that it is likely that the fetch of the
location attribute has cleared one the file systems contributing file system location attribute has cleared one the file systems
to the lease-migrated indication. contributing to the lease-migrated indication.
o In cases in which that happened, the thread cannot know whether o In cases in which that happened, the thread cannot know whether
the lease-migrated indication has been cleared and so it enters the lease-migrated indication has been cleared and so it enters
the completion/verification state and proceeds to issue a COMPOUND the completion/verification state and proceeds to issue a COMPOUND
to see if the LEASE_MOVED indication has been cleared. to see if the LEASE_MOVED indication has been cleared.
o When the discovery process is in the completion/verification o When the discovery process is in the completion/verification
state, if others request get a lease-migrated indication they note state, if others request get a lease-migrated indication they note
that it was received and the existence of such indications is used that it was received and the existence of such indications is used
when the request completes, as described below. when the request completes, as described below.
skipping to change at page 37, line 21 skipping to change at page 39, line 21
process. process.
It should be noted that the process described above is not guaranteed It should be noted that the process described above is not guaranteed
to terminate, as a long series of new migration events might to terminate, as a long series of new migration events might
continually delay the clearing of the LEASE_MOVED indication. To continually delay the clearing of the LEASE_MOVED indication. To
prevent unnecessary lease expiration, it is appropriate for clients prevent unnecessary lease expiration, it is appropriate for clients
to use the discovery of migrations to effect lease renewal to use the discovery of migrations to effect lease renewal
immediately, rather than waiting for clearing of the LEASE_MOVED immediately, rather than waiting for clearing of the LEASE_MOVED
indication when the complete set of migrations is available. indication when the complete set of migrations is available.
10.3. Overview of Client Response to NFS4ERR_MOVED (to be added) 10.3. Third sub-section within new section to be added to [RFC5661] to
be entitled "Overview of Client Response to NFS4ERR_MOVED"
This section outlines a way in which a client that receives This section outlines a way in which a client that receives
NFS4ERR_MOVED can effect transition recovery by using a new server or NFS4ERR_MOVED can effect transition recovery by using a new server or
server endpoint if one is available. As part of that process, it server endpoint if one is available. As part of that process, it
will determine: will determine:
o Whether the NFS4ERR_MOVED indicates migration has occurred, or o Whether the NFS4ERR_MOVED indicates migration has occurred, or
whether it indicates another sort of file system access transition whether it indicates another sort of file system access transition
as discussed in Section 7 above. as discussed in Section 7 above.
o In the case of migration, whether Transparent State Migration has o In the case of migration, whether Transparent State Migration has
occurred. occurred.
o Whether any state has been lost during the process of Transparent o Whether any state has been lost during the process of Transparent
State Migration. State Migration.
o Whether sessions have been transferred as part of Transparent o Whether sessions have been transferred as part of Transparent
State Migration. State Migration.
During the first phase of this process, the client proceeds to During the first phase of this process, the client proceeds to
examine location entries to find the initial network address it will examine file system location entries to find the initial network
use to continue access to the file system or its replacement. For address it will use to continue access to the file system or its
each location entry that the client examines, the process consists of replacement. For each location entry that the client examines, the
five steps: process consists of five steps:
1. Performing an EXCHANGE_ID directed at the location address. This 1. Performing an EXCHANGE_ID directed at the location address. This
operation is used to register the client-owner with the server, operation is used to register the client-owner with the server,
to obtain a client ID to be use subsequently to communicate with to obtain a client ID to be use subsequently to communicate with
it, to obtain that client ID's confirmation status, and to it, to obtain that client ID's confirmation status, and to
determine server_owner and scope for the purpose of determining determine server_owner and scope for the purpose of determining
if the entry is trunkable with that previously being used to if the entry is trunkable with that previously being used to
access the file system (i.e. that it represents another network access the file system (i.e. that it represents another network
access path to the same file system and can share locking state access path to the same file system and can share locking state
with it). with it).
skipping to change at page 39, line 14 skipping to change at page 41, line 16
2. In the case that the network address is session-trunkable with 2. In the case that the network address is session-trunkable with
one used previously a BIND_CONN_TO_SESSION is used to access that one used previously a BIND_CONN_TO_SESSION is used to access that
session using the new network address. Otherwise, or if the bind session using the new network address. Otherwise, or if the bind
operation fails, a CREATE_SESSION is done. operation fails, a CREATE_SESSION is done.
3. The verification procedure referred to in step 4 above is used. 3. The verification procedure referred to in step 4 above is used.
However, if it fails, the entry is ignored and the next available However, if it fails, the entry is ignored and the next available
entry is used. entry is used.
10.4. Obtaining Access to Sessions and State after Migration (to be 10.4. Fourth sub-section within new section to be added to [RFC5661] to
added) be entitled "Obtaining Access to Sessions and State after
Migration"
In the event that migration has occurred, migration recovery will In the event that migration has occurred, migration recovery will
involve determining whether Transparent State Migration has occurred. involve determining whether Transparent State Migration has occurred.
This decision is made based on the client ID returned by the This decision is made based on the client ID returned by the
EXCHANGE_ID and the reported confirmation status. EXCHANGE_ID and the reported confirmation status.
o If the client ID is an unconfirmed client ID not previously known o If the client ID is an unconfirmed client ID not previously known
to the client, then Transparent State Migration has not occurred. to the client, then Transparent State Migration has not occurred.
o If the client ID is a confirmed client ID previously known to the o If the client ID is a confirmed client ID previously known to the
skipping to change at page 41, line 5 skipping to change at page 43, line 7
o In a case in which Transparent State Migration has occurred, and o In a case in which Transparent State Migration has occurred, and
some lock state was lost (as shown by SEQ4_STATUS flags), existing some lock state was lost (as shown by SEQ4_STATUS flags), existing
stateids need to be checked for validity using TEST_STATEID, and stateids need to be checked for validity using TEST_STATEID, and
reclaim used to re-establish any that were not transferred. reclaim used to re-establish any that were not transferred.
For all of the cases above, RECLAIM_COMPLETE with an rca_one_fs value For all of the cases above, RECLAIM_COMPLETE with an rca_one_fs value
of TRUE needs to be done before normal use of the file system of TRUE needs to be done before normal use of the file system
including obtaining new locks for the file system. This applies even including obtaining new locks for the file system. This applies even
if no locks were lost and there was no need for any to be reclaimed. if no locks were lost and there was no need for any to be reclaimed.
10.5. Obtaining Access to Sessions and State after Network Address 10.5. Fifth sub-section within new section to be added to [RFC5661] to
Transfer (to be added) be entitled "Obtaining Access to Sessions and State after Network
Address Transfer"
The case in which there is a transfer to a new network address The case in which there is a transfer to a new network address
without migration is similar to that described in Section 10.4 above without migration is similar to that described in Section 10.4 above
in that there is a need to obtain access to needed sessions and in that there is a need to obtain access to needed sessions and
locking state. However, the details are simpler and will vary locking state. However, the details are simpler and will vary
depending on the type of trunking between the address receiving depending on the type of trunking between the address receiving
NFS4ERR_MOVED and that to which the transfer is to be made NFS4ERR_MOVED and that to which the transfer is to be made
To make a session available for use, a BIND_CONN_TO_SESSION should be To make a session available for use, a BIND_CONN_TO_SESSION should be
used to obtain access to the session previously in use. Only if this used to obtain access to the session previously in use. Only if this
fails, should a CREATE_SESSION be done. While this procedure mirrors fails, should a CREATE_SESSION be done. While this procedure mirrors
that in Section 10.4 above, there is an important difference in that that in Section 10.4 above, there is an important difference in that
preservation of the session is not purely optional but depends on the preservation of the session is not purely optional but depends on the
type of trunking. type of trunking.
Access to appropriate locking state should need no actions beyond Access to appropriate locking state should need no actions beyond
access to the session. However, the SEQ4_STATUS bits need to be access to the session. However, the SEQ4_STATUS bits need to be
checked for lost locking state, including the need to reclaim locks checked for lost locking state, including the need to reclaim locks
after a server reboot. after a server reboot.
11. Server Responsibilities Upon Migration (to be added) 11. New section to be added third after Section 11.11 of [RFC5661] to
be entitled "Server Responsibilities Upon Migration"
In the event of file system migration, when the client connects to In the event of file system migration, when the client connects to
the destination server, it needs to be able to provide the client the destination server, it needs to be able to provide the client
continued to access the files it had open on the source server. continued to access the files it had open on the source server.
There are two ways to provide this: There are two ways to provide this:
o By provision of an fs-specific grace period, allowing the client o By provision of an fs-specific grace period, allowing the client
the ability to reclaim its locks, in a fashion similar to what the ability to reclaim its locks, in a fashion similar to what
would have been done in the case of recovery from a server would have been done in the case of recovery from a server
restart. See Section 11.1 for a more complete discussion. restart. See Section 11.1 for a more complete discussion.
skipping to change at page 42, line 8 skipping to change at page 44, line 14
All the features described above can involve transfer of lock-related All the features described above can involve transfer of lock-related
information between source and destination servers. In some cases information between source and destination servers. In some cases
this transfer is a necessary part of the implementation while in this transfer is a necessary part of the implementation while in
other cases it is a helpful implementation aid which servers might or other cases it is a helpful implementation aid which servers might or
might not use. The sub-sections below discuss the information which might not use. The sub-sections below discuss the information which
would transferred but do not define the specifics of the transfer would transferred but do not define the specifics of the transfer
protocol. This is left as an implementation choice although protocol. This is left as an implementation choice although
standards in this area could be developed at a later time. standards in this area could be developed at a later time.
11.1. Server Responsibilities in Effecting State Reclaim after 11.1. First sub-section within new section to be added to [RFC5661] to
Migration (to be added) be entitled "Server Responsibilities in Effecting State Reclaim
after Migration"
In this case, destination server need have no knowledge of the locks In this case, destination server need have no knowledge of the locks
held on the source server, but relies on the clients to accurately held on the source server, but relies on the clients to accurately
report (via reclaim operations) the locks previously held, not report (via reclaim operations) the locks previously held, not
allowing new locks to be granted on migrated file system until the allowing new locks to be granted on migrated file system until the
grace period expires. grace period expires.
During this grace period clients have the opportunity to use reclaim During this grace period clients have the opportunity to use reclaim
operations to obtain locks for file system objects within the operations to obtain locks for file system objects within the
migrated file system, in the same way that they do when recovering migrated file system, in the same way that they do when recovering
skipping to change at page 42, line 44 skipping to change at page 45, line 5
for the transferred file system, the destination server will be for the transferred file system, the destination server will be
able to terminate the grace period once all such clients have able to terminate the grace period once all such clients have
reclaimed their locks, allowing normal locking activity to resume reclaimed their locks, allowing normal locking activity to resume
earlier than it would have otherwise. earlier than it would have otherwise.
o Locking summary information for individual clients (at various o Locking summary information for individual clients (at various
possible levels of detail) can detect some instances in which possible levels of detail) can detect some instances in which
clients do not accurately represent the locks held on the source clients do not accurately represent the locks held on the source
server. server.
11.2. Server Responsibilities in Effecting Transparent State Migration 11.2. Second sub-section within new section to be added to [RFC5661] to
(to be added) be entitled "Server Responsibilities in Effecting Transparent
State Migration"
The basic responsibility of the source server in effecting The basic responsibility of the source server in effecting
Transparent State Migration is to make available to the destination Transparent State Migration is to make available to the destination
server a description of each piece of locking state associated with server a description of each piece of locking state associated with
the file system being migrated. In addition to client id string and the file system being migrated. In addition to client id string and
verifier, the source server needs to provide, for each stateid: verifier, the source server needs to provide, for each stateid:
o The stateid including the current sequence value. o The stateid including the current sequence value.
o The associated client ID. o The associated client ID.
skipping to change at page 44, line 30 skipping to change at page 46, line 39
longer exists. longer exists.
o Sequencing of operations is no longer done using owner-based o Sequencing of operations is no longer done using owner-based
operation sequences numbers. Instead, sequencing is session- operation sequences numbers. Instead, sequencing is session-
based based
As a result, when sessions are not transferred, the techniques As a result, when sessions are not transferred, the techniques
discussed in Section 7.2 of [RFC7931] are adequate and will not be discussed in Section 7.2 of [RFC7931] are adequate and will not be
further discussed. further discussed.
11.3. Server Responsibilities in Effecting Session Transfer (to be 11.3. Third sub-section within new section to be added to [RFC5661] to
added) be entitled "Server Responsibilities in Effecting Session
Transfer"
The basic responsibility of the source server in effecting session The basic responsibility of the source server in effecting session
transfer is to make available to the destination server a description transfer is to make available to the destination server a description
of the current state of each slot with the session, including: of the current state of each slot with the session, including:
o The last sequence value received for that slot. o The last sequence value received for that slot.
o Whether there is cached reply data for the last request executed o Whether there is cached reply data for the last request executed
and, if so, the cached reply. and, if so, the cached reply.
skipping to change at page 47, line 28 skipping to change at page 49, line 34
rather than to a path to access a replica. rather than to a path to access a replica.
o In describing the appropriate value for a server to use for o In describing the appropriate value for a server to use for
fli_valid_for, it needs to be made clear that there is no need for fli_valid_for, it needs to be made clear that there is no need for
the client to frequently fetch the fs_locations_info value to be the client to frequently fetch the fs_locations_info value to be
prepared for shifts in trunking patterns. prepared for shifts in trunking patterns.
o Clarification of the rules for extensions of the fls_info needs to o Clarification of the rules for extensions of the fls_info needs to
be provided. The existing treatment reflects the extension model be provided. The existing treatment reflects the extension model
in effect at the time [RFC5661] was written, and need to be in effect at the time [RFC5661] was written, and need to be
updated in accord with the extension model described [RFC8178]. updated in accordance with the extension model described in
[RFC8178].
12.2. The Attribute fs_locations_info (as updated) 12.2. Updated Section 11.10 of [RFC5661] entitled "The Attribute
fs_locations_info"
The fs_locations_info attribute is intended as a more functional The fs_locations_info attribute is intended as a more functional
replacement for the fs_locations attribute which will continue to replacement for the fs_locations attribute which will continue to
exist and be supported. Clients can use it to get a more complete exist and be supported. Clients can use it to get a more complete
set of data about alternative file system locations, including set of data about alternative file system locations, including
additional network paths to access replicas in use and additional additional network paths to access replicas in use and additional
replicas. When the server does not support fs_locations_info, replicas. When the server does not support fs_locations_info,
fs_locations can be used to get a subset of the data. A server that fs_locations can be used to get a subset of the data. A server that
supports fs_locations_info MUST support fs_locations as well. supports fs_locations_info MUST support fs_locations as well.
skipping to change at page 48, line 33 skipping to change at page 50, line 42
The fs_locations_info attribute is structured similarly to the The fs_locations_info attribute is structured similarly to the
fs_locations attribute. A top-level structure (fs_locations_info4) fs_locations attribute. A top-level structure (fs_locations_info4)
contains the entire attribute including the root pathname of the file contains the entire attribute including the root pathname of the file
system and an array of lower-level structures that define replicas system and an array of lower-level structures that define replicas
that share a common rootpath on their respective servers. The lower- that share a common rootpath on their respective servers. The lower-
level structure in turn (fs_locations_item4) contains a specific level structure in turn (fs_locations_item4) contains a specific
pathname and information on one or more individual network access pathname and information on one or more individual network access
paths. For that last lowest level, fs_locations_info has an paths. For that last lowest level, fs_locations_info has an
fs_locations_server4 structure that contains per-server-replica fs_locations_server4 structure that contains per-server-replica
information in addition to the location entry. This per-server- information in addition to the file system location entry. This per-
replica information includes a nominally opaque array, fls_info, server-replica information includes a nominally opaque array,
within which specific pieces of information are located at the fls_info, within which specific pieces of information are located at
specific indices listed below. the specific indices listed below.
Two fs_location_server4 entries that are within different Two fs_location_server4 entries that are within different
fs_location_item4 structures are never trunkable, while two entries fs_location_item4 structures are never trunkable, while two entries
within in the same fs_location_item4 structure might or might not be within in the same fs_location_item4 structure might or might not be
trunkable. Two entries that are trunkable will have identical trunkable. Two entries that are trunkable will have identical
identity information, although, as noted above, the converse is not identity information, although, as noted above, the converse is not
the case. the case.
The attribute will always contain at least a single The attribute will always contain at least a single
fs_locations_server entry. Typically, there will be an entries with fs_locations_server entry. Typically, there will be an entries with
skipping to change at page 51, line 14 skipping to change at page 53, line 21
just referenced) and its successor location. Servers are strongly just referenced) and its successor location. Servers are strongly
urged to support this attribute on all file systems if they support urged to support this attribute on all file systems if they support
it on any file system. it on any file system.
The data presented in the fs_locations_info attribute may be obtained The data presented in the fs_locations_info attribute may be obtained
by the server in any number of ways, including specification by the by the server in any number of ways, including specification by the
administrator or by current protocols for transferring data among administrator or by current protocols for transferring data among
replicas and protocols not yet developed. NFSv4.1 only defines how replicas and protocols not yet developed. NFSv4.1 only defines how
this information is presented by the server to the client. this information is presented by the server to the client.
12.2.1. The fs_locations_server4 Structure (as updated) 12.2.1. Updated section 11.10.1 of [RFC5661] entitled "The
fs_locations_server4 Structure"
The fs_locations_server4 structure consists of the following items in The fs_locations_server4 structure consists of the following items in
addition to the fls_server field which specifies a network address or addition to the fls_server field which specifies a network address or
set of addresses to be used to access the specified file system. set of addresses to be used to access the specified file system.
Note that both of these items specify attributes of the file system Note that both of these items specify attributes of the file system
replica and should not be different when there are multiple replica and should not be different when there are multiple
fs_locations_server4 structures for the same replica, each specifying fs_locations_server4 structures for the same replica, each specifying
a network path to the chosen replica. a network path to the chosen replica.
o An indication of how up-to-date the file system is (fls_currency) o An indication of how up-to-date the file system is (fls_currency)
skipping to change at page 57, line 40 skipping to change at page 60, line 5
o The field at byte index FSLI4BX_WRITEORDER gives the order value o The field at byte index FSLI4BX_WRITEORDER gives the order value
to be used for writable access. to be used for writable access.
Depending on the potential need for write access by a given client, Depending on the potential need for write access by a given client,
one of the pairs of rank and order values is used. The read rank and one of the pairs of rank and order values is used. The read rank and
order should only be used if the client knows that only reading will order should only be used if the client knows that only reading will
ever be done or if it is prepared to switch to a different replica in ever be done or if it is prepared to switch to a different replica in
the event that any write access capability is required in the future. the event that any write access capability is required in the future.
12.2.2. The fs_locations_info4 Structure (as updated) 12.2.2. Updated Section 11.10.2 of [RFC5661] entitled "The
fs_locations_info4 Structure"
The fs_locations_info4 structure, encoding the fs_locations_info The fs_locations_info4 structure, encoding the fs_locations_info
attribute, contains the following: attribute, contains the following:
o The fli_flags field, which contains general flags that affect the o The fli_flags field, which contains general flags that affect the
interpretation of this fs_locations_info4 structure and all interpretation of this fs_locations_info4 structure and all
fs_locations_item4 structures within it. The only flag currently fs_locations_item4 structures within it. The only flag currently
defined is FSLI4IF_VAR_SUB. All bits in the fli_flags field that defined is FSLI4IF_VAR_SUB. All bits in the fli_flags field that
are not defined should always be returned as zero. are not defined should always be returned as zero.
skipping to change at page 59, line 5 skipping to change at page 61, line 15
Note that, because of the ability of the server to return Note that, because of the ability of the server to return
NFS4ERR_MOVED to change to use of different paths, when alternate NFS4ERR_MOVED to change to use of different paths, when alternate
trunked paths are available, there is generally no need to use low trunked paths are available, there is generally no need to use low
values of fli_valid_for in connection with the management of values of fli_valid_for in connection with the management of
alternate paths to the same replica. alternate paths to the same replica.
The FSLI4IF_VAR_SUB flag within fli_flags controls whether variable The FSLI4IF_VAR_SUB flag within fli_flags controls whether variable
substitution is to be enabled. See Section 12.2.3 for an explanation substitution is to be enabled. See Section 12.2.3 for an explanation
of variable substitution. of variable substitution.
12.2.3. The fs_locations_item4 Structure (as updated) 12.2.3. Updated Section 11.10.3 of [RFC5661] entitled "The
fs_locations_item4 Structure"
The fs_locations_item4 structure contains a pathname (in the field The fs_locations_item4 structure contains a pathname (in the field
fli_rootpath) that encodes the path of the target file system fli_rootpath) that encodes the path of the target file system
replicas on the set of servers designated by the included replicas on the set of servers designated by the included
fs_locations_server4 entries. The precise manner in which this fs_locations_server4 entries. The precise manner in which this
target location is specified depends on the value of the target location is specified depends on the value of the
FSLI4IF_VAR_SUB flag within the associated fs_locations_info4 FSLI4IF_VAR_SUB flag within the associated fs_locations_info4
structure. structure.
If this flag is not set, then fli_rootpath simply designates the If this flag is not set, then fli_rootpath simply designates the
skipping to change at page 61, line 5 skipping to change at page 63, line 12
substituted variables, the result is always a valid successor file substituted variables, the result is always a valid successor file
system instance to that from which a transition is occurring, i.e., system instance to that from which a transition is occurring, i.e.,
that the data is identical or represents a later image of a writable that the data is identical or represents a later image of a writable
file system. file system.
Note that when fli_rootpath is a null pathname (that is, one with Note that when fli_rootpath is a null pathname (that is, one with
zero components), the file system designated is at the root of the zero components), the file system designated is at the root of the
specified server, whether or not the FSLI4IF_VAR_SUB flag within the specified server, whether or not the FSLI4IF_VAR_SUB flag within the
associated fs_locations_info4 structure is set. associated fs_locations_info4 structure is set.
13. Changes to RFC5661 outside Section 11 13. Changes to [RFC5661] outside Section 11
Beside the major rework of Section 11, there are a number of related Beside the major rework of Section 11, there are a number of related
changes that are necessary: changes that are necessary:
o The summary that appeared in Section 1.7.3.3 of [RFC5661] needs to o The summary that appeared in Section 1.7.3.3 of [RFC5661] needs to
be revised to reflect the changes called for in Section 4 of the be revised to reflect the changes called for in Section 4 of the
current document. The updated summary appears as Section 13.1 current document. The updated summary appears as Section 13.1
below. below.
o The discussion of server scope which appeared in Section 2.10.4 of o The discussion of server scope which appeared in Section 2.10.4 of
skipping to change at page 62, line 10 skipping to change at page 64, line 19
the rca_one_fs and how the server is to deal with inappropriate the rca_one_fs and how the server is to deal with inappropriate
values of this argument. Because the resulting confusion raises values of this argument. Because the resulting confusion raises
interoperability issues, a new treatment of RECLAIM_COMPLETE is interoperability issues, a new treatment of RECLAIM_COMPLETE is
necessary and it appears in Section 15 below while the specific necessary and it appears in Section 15 below while the specific
differences between it and the treatment within [RFC5661] are differences between it and the treatment within [RFC5661] are
discussed in Section 13.6 below. In addition, the definitions of discussed in Section 13.6 below. In addition, the definitions of
the reclaim-related errors receive an updated treatment in the reclaim-related errors receive an updated treatment in
Section 13.7 to reflect the fact that there are multiple contexts Section 13.7 to reflect the fact that there are multiple contexts
for lock reclaim operations. for lock reclaim operations.
13.1. (Introduction to) Multi-Server Namespace (as updated) 13.1. Updated section 1.7.3.3 of [RFC5661] to be retitled "Introduction
to Multi-Server Namespace"
NFSv4.1 contains a number of features to allow implementation of NFSv4.1 contains a number of features to allow implementation of
namespaces that cross server boundaries and that allow and facilitate namespaces that cross server boundaries and that allow and facilitate
a non-disruptive transfer of support for individual file systems a non-disruptive transfer of support for individual file systems
between servers. They are all based upon attributes that allow one between servers. They are all based upon attributes that allow one
file system to specify alternate, additional, and new location file system to specify alternate, additional, and new location
information which specifies how the client may access to access that information which specifies how the client may access to access that
file system. file system.
These attributes can be used to provide for individual active file These attributes can be used to provide for individual active file
systems: systems:
o Alternate network addresses to access the current file system o Alternate network addresses to access the current file system
instance. instance.
o The locations of alternate file system instances or replicas to be o The locations of alternate file system instances or replicas to be
used in the event that the current file system instance becomes used in the event that the current file system instance becomes
unavailable. unavailable.
These attributes may be used together with the concept of absent file These file system location attributes may be used together with the
systems, in which a position in the server namespace is associated concept of absent file systems, in which a position in the server
with locations on other servers without there being any corresponding namespace is associated with locations on other servers without there
file system instance on the current server. being any corresponding file system instance on the current server.
o Location attributes may be used with absent file systems to o These attributes may be used with absent file systems to implement
implement referrals whereby one server may direct the client to a referrals whereby one server may direct the client to a file
file system provided by another server. This allows extensive system provided by another server. This allows extensive multi-
multi-server namespaces to be constructed. server namespaces to be constructed.
o Location attributes may be provided when a previously present file o These attributes may be provided when a previously present file
system becomes absent. This allows non-disruptive migration of system becomes absent. This allows non-disruptive migration of
file systems to alternate servers. file systems to alternate servers.
13.2. Server Scope (as updated) 13.2. Updated Section 2.10.4 of [RFC5661] entitled "Server Scope"
Servers each specify a server scope value in the form of an opaque Servers each specify a server scope value in the form of an opaque
string eir_server_scope returned as part of the results of an string eir_server_scope returned as part of the results of an
EXCHANGE_ID operation. The purpose of the server scope is to allow a EXCHANGE_ID operation. The purpose of the server scope is to allow a
group of servers to indicate to clients that a set of servers sharing group of servers to indicate to clients that a set of servers sharing
the same server scope value has arranged to use compatible values of the same server scope value has arranged to use compatible values of
otherwise opaque identifiers. Thus, the identifiers generated by two otherwise opaque identifiers. Thus, the identifiers generated by two
servers within that set can be assumed compatible so that, in some servers within that set can be assumed compatible so that, in some
cases, identifiers by one server in that set that set may be cases, identifiers by one server in that set that set may be
presented to another server of the same scope. presented to another server of the same scope.
skipping to change at page 65, line 7 skipping to change at page 67, line 18
with it or it might not be present at the server. In the latter with it or it might not be present at the server. In the latter
case, it might have been relocated or migrated to another server, case, it might have been relocated or migrated to another server,
or it might have never been present. The client may obtain or it might have never been present. The client may obtain
information regarding access to the file system location by information regarding access to the file system location by
obtaining the "fs_locations" or "fs_locations_info" attribute for obtaining the "fs_locations" or "fs_locations_info" attribute for
the current filehandle. For further discussion, refer to the current filehandle. For further discussion, refer to
Section 11 of [RFC5661], as modified by the current document. Section 11 of [RFC5661], as modified by the current document.
13.4. Revised Discussion of Server_owner changes 13.4. Revised Discussion of Server_owner changes
Because of likely problems with the treatment of such changes, a Section 2.10.5 of [RFC5661] discusses the issue of possible
confusing paragraph which appear at the end of Section 2.5.10 if server_owner changes as follows:
[RFC5661], which simply says that such changes need to be dealt with,
is to be replaced by the material below. The client should be prepared for the possibility that
eir_server_owner values may be different on subsequent EXCHANGE_ID
requests made to the same network address, as a result of various
sorts of reconfiguration events. When this happens and the
changes result in the invalidation of previously valid forms of
trunking, the client should cease to use those forms, either by
dropping connections or by adding sessions. For a discussion of
lock reclaim as it relates to such reconfiguration events, see
Section 8.4.2.1
While this paragraph is literally true in that such reconfiguration
events can happen and clients have to deal with them, it is confusing
in that it can be read as suggesting that clients have to deal with
them without disruption, which in general is impossible. This has
led to confusion especially since the text is not very clear about
the actions that might need to be done since:
o The cases of change which are very disruptive (e.g. change if
server scope) are not sufficiently distinguished from those that
simply involve a change of trunking modes (i.e. change
server_owner minor id)
o There is an undue focus on the effect of such changes as they
affect the comparison with corresponding value from other servers,
without fully dealing with the issue that result from value
discontinuity within a single server.
Because of these issues the paragraph which appears at the end of
Section 2.10.5 needs to be replaced by the material below.
It is always possible that, as a result of various sorts of It is always possible that, as a result of various sorts of
reconfiguration events, eir_server_scope and eir_server_owner reconfiguration events, eir_server_scope and eir_server_owner
values may be different on subsequent EXCHANGE_ID requests made to values may be different on subsequent EXCHANGE_ID requests made to
the same network address. the same network address.
In most cases such reconfiguration events will be disruptive and In most cases such reconfiguration events will be disruptive and
indicate that an IP address formerly connected to one server is indicate that an IP address formerly connected to one server is
now connected to an entirely different one. now connected to an entirely different one.
Some guidelines on client handling of such situations follow: Some guidelines on client handling of such situations follow:
* When eir_server_scope changes, the client has no assurance that * When eir_server_scope changes, the client has no assurance that
any id's it obtained previously (e.g. file handles) can be any id's it obtained previously (e.g. file handles, state ids,
validly used on the new server, and, even if the new server client ids) can be validly used on the new server, and, even if
accepts them, there is no assurance that this is not due to the new server accepts them, there is no assurance that this is
accident. Thus it is best to treat all such state as lost/ not due to accident. Thus it is best to treat all such state
stale although a client may assume that the probability of as lost/stale although a client may assume that the probability
inadvertent acceptance is low and treat this situation as of inadvertent acceptance is low and treat this situation as
within the next case. within the next case.
* When eir_server_scope remains the same and * When eir_server_scope remains the same and
eir_server_owner.so_major_id changes, the client can use eir_server_owner.so_major_id changes, the client can use the
filehandles it has and attempt reclaims. It may find that filehandles it has, consider its locking state lost, and
these are now stale but if NFS4ERR_STALE is not received, he attempt to reclaim or otherwise re-obtain its locks. It may
can proceed to reclaim his opens. find that its file handle are now stale but if NFS4ERR_STALE is
not received, it can proceed to reclaim or otherwise re-obtain
its open locking state.
* When eir_server_scope and eir_server_owner.so_major_id remain * When eir_server_scope and eir_server_owner.so_major_id remain
the same, the client has to use the now-current values of the same, the client has to use the now-current values of
eir_server_owner.so_minor_id in deciding on appropriate forms eir_server_owner.so_minor_id in deciding on appropriate forms
of trunking. of trunking. This may result in connections being dropped or
new sessions being created.
13.5. Revision to Treatment of EXCHANGE_ID 13.5. Revision to Treatment of EXCHANGE_ID
There are a number of issues in the original treatment of EXCHANGE_ID There are a number of issues in the original treatment of EXCHANGE_ID
(in [RFC5661]) that cause problems for Transparent State Migration (in [RFC5661]) that cause problems for Transparent State Migration
and for the transfer of access between different network access paths and for the transfer of access between different network access paths
to the same file system instance. to the same file system instance.
These issues arise from the fact that this treatment was written: These issues arise from the fact that this treatment was written:
skipping to change at page 67, line 20 skipping to change at page 70, line 12
o In a number of places the text is more explicit about the purpose o In a number of places the text is more explicit about the purpose
of rca_one_fs and its connection to file system migration. of rca_one_fs and its connection to file system migration.
o There is a discussion of situations in which either form of o There is a discussion of situations in which either form of
RECLAIM_COMPLETE would need to be done. RECLAIM_COMPLETE would need to be done.
o There is a discussion of interoperability issues that result from o There is a discussion of interoperability issues that result from
implementations that may have arisen due to the lack of clarity of implementations that may have arisen due to the lack of clarity of
the previous treatment of RECLAIM_COMPLETE. the previous treatment of RECLAIM_COMPLETE.
13.7. Reclaim Errors (as updated) 13.7. Updated Section 15.1.9 of [RFC5661] entitled "Reclaim Errors"
These errors relate to the process of reclaiming locks after a server These errors relate to the process of reclaiming locks after a server
restart or in connection with the migration of a file system (i.e. in restart or in connection with the migration of a file system (i.e. in
the case in which rca_one_fs is TRUE). the case in which rca_one_fs is TRUE).
13.7.1. NFS4ERR_COMPLETE_ALREADY (as updated; Error Code 10054) 13.7.1. Updated Section 15.1.9.1 of [RFC5661] entitled
"NFS4ERR_COMPLETE_ALREADY (Error Code 10054)"
The client previously sent a successful RECLAIM_COMPLETE operation The client previously sent a successful RECLAIM_COMPLETE operation
specifying the same scope, whether that scope is global or for the specifying the same scope, whether that scope is global or for the
same file system in the case of a per-fs RECLAIM_COMPLETE. An same file system in the case of a per-fs RECLAIM_COMPLETE. An
additional RECLAIM_COMPLETE operation is not necessary and results in additional RECLAIM_COMPLETE operation is not necessary and results in
this error. this error.
13.7.2. NFS4ERR_GRACE (as updated; Error Code 10013) 13.7.2. Updated Section 15.1.9.2 of [RFC5661] entitled "NFS4ERR_GRACE
(Error Code 10013)"
The server was in its recovery or grace period, with regard to the The server was in its recovery or grace period, with regard to the
file system object for which the lock was requested. The locking file system object for which the lock was requested. The locking
request was not a reclaim request and so could not be granted during request was not a reclaim request and so could not be granted during
that period. that period.
13.7.3. NFS4ERR_NO_GRACE (as updated; Error Code 10033) 13.7.3. Updated Section 15.1.9.3 of [RFC5661] entitled
"NFS4ERR_NO_GRACE (Error Code 10033)"
A reclaim of client state was attempted in circumstances in which the A reclaim of client state was attempted in circumstances in which the
server cannot guarantee that conflicting state has not been provided server cannot guarantee that conflicting state has not been provided
to another client. This can occur because the reclaim has been done to another client. This can occur because the reclaim has been done
outside of a grace period of implemented by the server, after the outside of a grace period of implemented by the server, after the
client has done a RECLAIM_COMPLETE operation which ends its ability client has done a RECLAIM_COMPLETE operation which ends its ability
to reclaim the requested lock, or because previous operations have to reclaim the requested lock, or because previous operations have
created a situation in which the server is not able to determine that created a situation in which the server is not able to determine that
a reclaim-interfering edge condition does not exist. a reclaim-interfering edge condition does not exist.
13.7.4. NFS4ERR_RECLAIM_BAD (as updated; Error Code 10034) 13.7.4. Updated Section 15.1.9.4 of [RFC5661] entitled
"NFS4ERR_RECLAIM_BAD (Error Code 10034)"
The server has determined that a reclaim attempted by the client is The server has determined that a reclaim attempted by the client is
not valid, i.e. the lock specified as being reclaimed could not not valid, i.e. the lock specified as being reclaimed could not
possibly have existed before the server restart or file system possibly have existed before the server restart or file system
migration event. A server is not obliged to make this determination migration event. A server is not obliged to make this determination
and will typically rely on the client to only reclaim locks that the and will typically rely on the client to only reclaim locks that the
client was granted prior to restart or file system migration. client was granted prior to restart or file system migration.
However, when a server does have reliable information to enable it However, when a server does have reliable information to enable it
make this determination, this error indicates that the reclaim has make this determination, this error indicates that the reclaim has
been rejected as invalid. This is as opposed to the error been rejected as invalid. This is as opposed to the error
NFS4ERR_RECLAIM_CONFLICT (see Section 13.7.5) where the server can NFS4ERR_RECLAIM_CONFLICT (see Section 13.7.5) where the server can
only determine that there has been an invalid reclaim, but cannot only determine that there has been an invalid reclaim, but cannot
determine which request is invalid. determine which request is invalid.
13.7.5. NFS4ERR_RECLAIM_CONFLICT (as updated; Error Code 10035) 13.7.5. Updated Section 15.1.9.5 of [RFC5661] entitled
"NFS4ERR_RECLAIM_CONFLICT (Error Code 10035)"
The reclaim attempted by the client has encountered a conflict and The reclaim attempted by the client has encountered a conflict and
cannot be satisfied. Potentially indicates a misbehaving client, cannot be satisfied. Potentially indicates a misbehaving client,
although not necessarily the one receiving the error. The although not necessarily the one receiving the error. The
misbehavior might be on the part of the client that established the misbehavior might be on the part of the client that established the
lock with which this client conflicted. See also Section 13.7.4 for lock with which this client conflicted. See also Section 13.7.4 for
the related error, NFS4ERR_RECLAIM_BAD. the related error, NFS4ERR_RECLAIM_BAD.
14. Operation 42: EXCHANGE_ID - Instantiate Client ID (as updated) 14. Updated Section 18.35 of [RFC5661] entitled "Operation 42:
EXCHANGE_ID - Instantiate Client ID"
The EXCHANGE_ID exchanges long-hand client and server identifiers The EXCHANGE_ID exchanges long-hand client and server identifiers
(owners), and provides access to a client ID, creating one if (owners), and provides access to a client ID, creating one if
necessary. This client ID becomes associated with the connection on necessary. This client ID becomes associated with the connection on
which the operation is done, so that it is available when a which the operation is done, so that it is available when a
CREATE_SESSION is done or when the connection is used to issue a CREATE_SESSION is done or when the connection is used to issue a
request on an existing session associated with the current client. request on an existing session associated with the current client.
14.1. ARGUMENT 14.1. Updated Section 18.35.1 of [RFC5661] entitled "ARGUMENT"
<CODE BEGINS> <CODE BEGINS>
const EXCHGID4_FLAG_SUPP_MOVED_REFER = 0x00000001; const EXCHGID4_FLAG_SUPP_MOVED_REFER = 0x00000001;
const EXCHGID4_FLAG_SUPP_MOVED_MIGR = 0x00000002; const EXCHGID4_FLAG_SUPP_MOVED_MIGR = 0x00000002;
const EXCHGID4_FLAG_BIND_PRINC_STATEID = 0x00000100; const EXCHGID4_FLAG_BIND_PRINC_STATEID = 0x00000100;
const EXCHGID4_FLAG_USE_NON_PNFS = 0x00010000; const EXCHGID4_FLAG_USE_NON_PNFS = 0x00010000;
const EXCHGID4_FLAG_USE_PNFS_MDS = 0x00020000; const EXCHGID4_FLAG_USE_PNFS_MDS = 0x00020000;
skipping to change at page 69, line 44 skipping to change at page 72, line 41
struct EXCHANGE_ID4args { struct EXCHANGE_ID4args {
client_owner4 eia_clientowner; client_owner4 eia_clientowner;
uint32_t eia_flags; uint32_t eia_flags;
state_protect4_a eia_state_protect; state_protect4_a eia_state_protect;
nfs_impl_id4 eia_client_impl_id<1>; nfs_impl_id4 eia_client_impl_id<1>;
}; };
<CODE ENDS> <CODE ENDS>
14.2. RESULT 14.2. Updated Section 18.35.2 of [RFC5661] entitled "RESULT"
<CODE BEGINS> <CODE BEGINS>
struct ssv_prot_info4 { struct ssv_prot_info4 {
state_protect_ops4 spi_ops; state_protect_ops4 spi_ops;
uint32_t spi_hash_alg; uint32_t spi_hash_alg;
uint32_t spi_encr_alg; uint32_t spi_encr_alg;
uint32_t spi_ssv_len; uint32_t spi_ssv_len;
uint32_t spi_window; uint32_t spi_window;
gsshandle4_t spi_handles<>; gsshandle4_t spi_handles<>;
}; };
skipping to change at page 70, line 44 skipping to change at page 73, line 44
union EXCHANGE_ID4res switch (nfsstat4 eir_status) { union EXCHANGE_ID4res switch (nfsstat4 eir_status) {
case NFS4_OK: case NFS4_OK:
EXCHANGE_ID4resok eir_resok4; EXCHANGE_ID4resok eir_resok4;
default: default:
void; void;
}; };
<CODE ENDS> <CODE ENDS>
14.3. DESCRIPTION 14.3. Updated Section 18.35.3 of [RFC5661] entitled "DESCRIPTION"
The client uses the EXCHANGE_ID operation to register a particular The client uses the EXCHANGE_ID operation to register a particular
client_owner with the server. However, when the client_owner has client_owner with the server. However, when the client_owner has
been already been registered by other means (e.g. Transparent State been already been registered by other means (e.g. Transparent State
Migration), the client may still use EXCHANGE_ID to obtain the client Migration), the client may still use EXCHANGE_ID to obtain the client
ID assigned previously. ID assigned previously.
The client ID returned from this operation will be associated with The client ID returned from this operation will be associated with
the connection on which the EXHANGE_ID is received and will serve as the connection on which the EXHANGE_ID is received and will serve as
a parent object for sessions created by the client on this connection a parent object for sessions created by the client on this connection
skipping to change at page 80, line 45 skipping to change at page 83, line 45
peer's manifesting a particular allowed behavior based on an peer's manifesting a particular allowed behavior based on an
implementation identifier but are required to interoperate as implementation identifier but are required to interoperate as
specified elsewhere in the protocol specification. specified elsewhere in the protocol specification.
Because it is possible that some implementations might violate the Because it is possible that some implementations might violate the
protocol specification and interpret the identity information, protocol specification and interpret the identity information,
implementations MUST provide facilities to allow the NFSv4 client and implementations MUST provide facilities to allow the NFSv4 client and
server be configured to set the contents of the nfs_impl_id server be configured to set the contents of the nfs_impl_id
structures sent to any specified value. structures sent to any specified value.
14.4. IMPLEMENTATION 14.4. Updated Section 18.35.4 of [RFC5661] entitled "IMPLEMENTATION"
A server's client record is a 5-tuple: A server's client record is a 5-tuple:
1. co_ownerid 1. co_ownerid
The client identifier string, from the eia_clientowner The client identifier string, from the eia_clientowner
structure of the EXCHANGE_ID4args structure. structure of the EXCHANGE_ID4args structure.
2. co_verifier: 2. co_verifier:
A client-specific value used to indicate incarnations (where a A client-specific value used to indicate incarnations (where a
skipping to change at page 86, line 44 skipping to change at page 89, line 44
If EXCHGID4_FLAG_UPD_CONFIRMED_REC_A is set, and the server If EXCHGID4_FLAG_UPD_CONFIRMED_REC_A is set, and the server
has the following confirmed record, then this request is an has the following confirmed record, then this request is an
illegal attempt at an update by an unauthorized principal. illegal attempt at an update by an unauthorized principal.
{ ownerid_arg, verifier_arg, old_principal_arg, clientid_ret, { ownerid_arg, verifier_arg, old_principal_arg, clientid_ret,
confirmed } confirmed }
The server returns NFS4ERR_PERM and leaves the client record The server returns NFS4ERR_PERM and leaves the client record
intact. intact.
15. Operation 58: RECLAIM_COMPLETE - Indicates Reclaims Finished (as 15. Updated Section 18.51 of [RFC5661] entitled "Operation 58:
updated) RECLAIM_COMPLETE - Indicates Reclaims Finished"
15.1. ARGUMENT 15.1. Updated Section 18.51.1 of [RFC5661] entitled "ARGUMEBNT"
<CODE BEGINS> <CODE BEGINS>
struct RECLAIM_COMPLETE4args { struct RECLAIM_COMPLETE4args {
/* /*
* If rca_one_fs TRUE, * If rca_one_fs TRUE,
* *
* CURRENT_FH: object in * CURRENT_FH: object in
* file system reclaim is * file system reclaim is
* complete for. * complete for.
*/ */
bool rca_one_fs; bool rca_one_fs;
}; };
<CODE ENDS> <CODE ENDS>
15.2. RESULTS 15.2. Updated Section 18.51.2 of [RFC5661] entitled "RESULTS"
<CODE BEGINS> <CODE BEGINS>
struct RECLAIM_COMPLETE4res { struct RECLAIM_COMPLETE4res {
nfsstat4 rcr_status; nfsstat4 rcr_status;
}; };
<CODE ENDS> <CODE ENDS>
15.3. DESCRIPTION 15.3. Updated Section 18.51.3 of [RFC5661] entitled "DESCRIPTION"
A RECLAIM_COMPLETE operation is used to indicate that the client has A RECLAIM_COMPLETE operation is used to indicate that the client has
reclaimed all of the locking state that it will recover using reclaimed all of the locking state that it will recover using
reclaim, when it is recovering state due to either a server restart reclaim, when it is recovering state due to either a server restart
or the migration of a file system to another server. There are two or the migration of a file system to another server. There are two
types of RECLAIM_COMPLETE operations: types of RECLAIM_COMPLETE operations:
o When rca_one_fs is FALSE, a global RECLAIM_COMPLETE is being done. o When rca_one_fs is FALSE, a global RECLAIM_COMPLETE is being done.
This indicates that recovery of all locks that the client held on This indicates that recovery of all locks that the client held on
the previous server instance have been completed. The current the previous server instance have been completed. The current
skipping to change at page 89, line 5 skipping to change at page 92, line 5
locks that it does not own, and so has no right to reclaim. See locks that it does not own, and so has no right to reclaim. See
Section 8.4.3 of [RFC5661] for a discussion of edge conditions Section 8.4.3 of [RFC5661] for a discussion of edge conditions
related to lock reclaim. related to lock reclaim.
By sending a RECLAIM_COMPLETE, the client indicates readiness to By sending a RECLAIM_COMPLETE, the client indicates readiness to
proceed to do normal non-reclaim locking operations. The client proceed to do normal non-reclaim locking operations. The client
should be aware that such operations may temporarily result in should be aware that such operations may temporarily result in
NFS4ERR_GRACE errors until the server is ready to terminate its grace NFS4ERR_GRACE errors until the server is ready to terminate its grace
period. period.
15.4. IMPLEMENTATION 15.4. Updated Section 18.51.4 of [RFC5661] entitled "IMPLEMENTATION"
Servers will typically use the information as to when reclaim Servers will typically use the information as to when reclaim
activity is complete to reduce the length of the grace period. When activity is complete to reduce the length of the grace period. When
the server maintains in persistent storage a list of clients that the server maintains in persistent storage a list of clients that
might have had locks, it is able to use the fact that all such might have had locks, it is able to use the fact that all such
clients have done a RECLAIM_COMPLETE to terminate the grace period clients have done a RECLAIM_COMPLETE to terminate the grace period
and begin normal operations (i.e., grant requests for new locks) and begin normal operations (i.e., grant requests for new locks)
sooner than it might otherwise. sooner than it might otherwise.
Latency can be minimized by doing a RECLAIM_COMPLETE as part of the Latency can be minimized by doing a RECLAIM_COMPLETE as part of the
skipping to change at page 90, line 34 skipping to change at page 93, line 34
In light of this, the following considerations should be taken In light of this, the following considerations should be taken
note of: note of:
o When DNS is used to convert server named to addresses and o When DNS is used to convert server named to addresses and
DNSSEC [RFC4033] is not available, the validity of the network DNSSEC [RFC4033] is not available, the validity of the network
addresses returned cannot be relied upon. However, when the addresses returned cannot be relied upon. However, when the
client uses RPCSEC_GSS to access the designated server, it is client uses RPCSEC_GSS to access the designated server, it is
possible for mutual authentication to discover invalid server possible for mutual authentication to discover invalid server
addresses provided. addresses provided.
o The fetching of attributes containing location information o The fetching of attributes containing file system location
SHOULD be performed using RPCSEC_GSS with integrity protection, information SHOULD be performed using RPCSEC_GSS with integrity
as previously explained in the Security Considerations section protection, as previously explained in the Security
of [RFC5661]. It is important to note here that a client Considerations section of [RFC5661]. It is important to note
making a request of this sort without using RPCSEC_GSS here that a client making a request of this sort without using
including integrity protection needs be aware of the negative RPCSEC_GSS including integrity protection needs be aware of the
consequences of doing so, which can lead to invalid host names negative consequences of doing so, which can lead to invalid
or network addresses being returned. In light of this, the host names or network addresses being returned. In light of
client needs to recognize that using such returned location this, the client needs to recognize that using such returned
information to access an NFSv4 server without use of RPCSEC_GSS location information to access an NFSv4 server without use of
(i.e. by using AUTH_SYS) poses dangers as it can result in the RPCSEC_GSS (i.e. by using AUTH_SYS) poses dangers as it can
client interacting with an unverified network address posing as result in the client interacting with an unverified network
an NFSv4 server. address posing as an NFSv4 server.
o Despite the fact that it is a REQUIREMENT (of [RFC5661]) that o Despite the fact that it is a REQUIREMENT (of [RFC5661]) that
"implementations" provide "support" for use of RPCSEC_GSS, it "implementations" provide "support" for use of RPCSEC_GSS, it
cannot be assumed that use of RPCSEC_GSS is always available cannot be assumed that use of RPCSEC_GSS is always available
between any particular client-server pair. between any particular client-server pair.
o When a client has the network addresses of a server but not the o When a client has the network addresses of a server but not the
associated host names, that would interfere with its ability to associated host names, that would interfere with its ability to
use RPCSEC_GSS. use RPCSEC_GSS.
In light of the above, a server should present location entries In light of the above, a server should present file system
that correspond to file systems on other servers using a host location entries that correspond to file systems on other servers
name. This would allow the client to interrogate the fs_locations using a host name. This would allow the client to interrogate the
on the destination server to obtain trunking information (as well fs_locations on the destination server to obtain trunking
as replica information) using RPCSEC_GSS with integrity, information (as well as replica information) using RPCSEC_GSS with
validating the name provided while assuring that the response has integrity, validating the name provided while assuring that the
not been corrupted. response has not been corrupted.
When RPCSEC_GSS is not available on a server, the client needs to When RPCSEC_GSS is not available on a server, the client needs to
be aware of the fact that the location entries are subject to be aware of the fact that the location entries are subject to
corruption and cannot be relied upon. In the case of a client corruption and cannot be relied upon. In the case of a client
being directed to another server after NFS4ERR_MOVED, this could being directed to another server after NFS4ERR_MOVED, this could
vitiate the authentication provided by the use of RPCSEC_GSS on vitiate the authentication provided by the use of RPCSEC_GSS on
the destination. Even when RPCSEC_GSS authentication is available the destination. Even when RPCSEC_GSS authentication is available
on the destination, the server might validly represent itself as on the destination, the server might validly represent itself as
the server to which the client was erroneously directed. Without the server to which the client was erroneously directed. Without
a way to decide whether the server is a valid one, the client can a way to decide whether the server is a valid one, the client can
only determine, using RPCSEC_GSS, that the server corresponds to only determine, using RPCSEC_GSS, that the server corresponds to
the name provided, with no basis for trusting that server. As a the name provided, with no basis for trusting that server. As a
result, the client should not use such unverified location entries result, the client should not use such unverified location entries
as a basis for migration, even though RPCSEC_GSS might be as a basis for migration, even though RPCSEC_GSS might be
available on the destination. available on the destination.
When a location attribute is fetched upon connecting with an NFS When a file system location attribute is fetched upon connecting
server, it SHOULD, as stated above, be done using RPCSEC_GSS with with an NFS server, it SHOULD, as stated above, be done using
integrity protection. When this not possible, it is generally RPCSEC_GSS with integrity protection. When this not possible, it
best for the client to ignore trunking and replica information or is generally best for the client to ignore trunking and replica
simply not fetch the location information for these purposes. information or simply not fetch the location information for these
purposes.
When location information cannot be verified, it can be subjected When location information cannot be verified, it can be subjected
to additional filtering to prevent the client from being to additional filtering to prevent the client from being
inappropriately directed. For example, if a range of network inappropriately directed. For example, if a range of network
addresses can be determined that assure that the servers and addresses can be determined that assure that the servers and
clients using AUTH_SYS are subject to the appropriate set of clients using AUTH_SYS are subject to the appropriate set of
constrains (e.g. physical network isolation, administrative constrains (e.g. physical network isolation, administrative
controls on the operating systems used), then network addresses in controls on the operating systems used), then network addresses in
the appropriate range can be used with others discarded or the appropriate range can be used with others discarded or
restricted in their use of AUTH_SYS. restricted in their use of AUTH_SYS.
skipping to change at page 95, line 33 skipping to change at page 98, line 40
To summarize: To summarize:
o There are seventeen explanatory sections. o There are seventeen explanatory sections.
o There are thirty-seven replacement sections. o There are thirty-seven replacement sections.
o There are eightteen additional sections. o There are eightteen additional sections.
o There are three editing sections. o There are three editing sections.
Appendix B. Updates to RFC5661 Appendix B. Updates to [RFC5661]
In this appendix, we proceed through [RFC5661] identifying sections In this appendix, we proceed through [RFC5661] identifying sections
as unchanged, modified, deleted, or replaced and indicating where as unchanged, modified, deleted, or replaced and indicating where
additional sections from the current document would appear in an additional sections from the current document would appear in an
eventual consolidated description of NFSv4.1. In this presentation, eventual consolidated description of NFSv4.1. In this presentation,
when section X is referred to, it denotes that section plus all when section X is referred to, it denotes that section plus all
included subsections. When it is necessary to refer to the part of a included subsections. When it is necessary to refer to the part of a
section outside any included subsections, the exclusion is noted section outside any included subsections, the exclusion is noted
explicitly. explicitly.
skipping to change at page 96, line 26 skipping to change at page 99, line 32
document. document.
o Sections 11.2, 11.3, 11.3.1, and 11.3.2 are unchanged. o Sections 11.2, 11.3, 11.3.1, and 11.3.2 are unchanged.
o Section 11.4 is replaced by Section 4.5 from the current o Section 11.4 is replaced by Section 4.5 from the current
document. For details regarding subsections see below. document. For details regarding subsections see below.
o New sections corresponding to Sections 4.5.1 through 4.5.3 o New sections corresponding to Sections 4.5.1 through 4.5.3
from the current document appear next. from the current document appear next.
o Section 11.4.1 is replaced by Section 4.5.4 o Section 11.4.1 is replaced by Section 4.5.4 from the current
document.
o Section 11.4.2 is replaced by Section 4.5.5 o Section 11.4.2 is replaced by Section 4.5.5 from the current
document.
o Section 11.4.3 is replaced by Section 4.5.6 o Section 11.4.3 is replaced by Section 4.5.6 from the current
document.
o A new section corresponding to Section 4.5.7 from the o A new section corresponding to Section 4.5.7 from the
current document appears next. current document appears next.
o Section 11.5 is to be deleted. o Section 11.5 is to be deleted.
o Section 11.6 is unchanged. o Section 11.6 is unchanged.
o New sections corresponding to Sections 6 and 7 from the current o New sections corresponding to Sections 6 and 7 from the current
document appear next. document appear next.
o Section 11.7 is replaced by Section 8 from the current o Section 11.7 is replaced by Section 8 from the current
document. For details regarding subsections see below. document. For details regarding subsections see below.
o Section 11.7.1 is replaced by Section 8.1 o Section 11.7.1 is replaced by Section 8.1 from the current
document.
o Sections 11.7.2, 11.7.2.1, and 11.7.2.2 are deleted. o Sections 11.7.2, 11.7.2.1, and 11.7.2.2 are deleted.
o Section 11.7.3 is replaced by Section 8.2 o Section 11.7.3 is replaced by Section 8.2 from the current
document.
o Section 11.7.4 is replaced by Section 8.3 from the current
document.
o Section 11.7.4 is replaced by Section 8.3
o Sections 11.7.5 and 11.7.5.1 are replaced by Sections 8.4 o Sections 11.7.5 and 11.7.5.1 are replaced by Sections 8.4
and 8.4.1 respectively. and 8.4.1 respectively, from the current document.
o Section 11.7.6 is replaced by Section 8.5 o Section 11.7.6 is replaced by Section 8.5 from the current
document.
o Section 11.7.7, exclusive of subsections, is replaced by o Section 11.7.7, exclusive of subsections, is replaced by
Section 8.9. Sections 11.7.7.1 and 11.7.72 are unchanged. Section 8.9 from the current document. Sections 11.7.7.1
and 11.7.7.2 are unchanged.
o Section 11.7.8 is replaced by Section 8.6 o Section 11.7.8 is replaced by Section 8.6 from the current
document.
o Section 11.7.9 is replaced by Section 8.7 o Section 11.7.9 is replaced by Section 8.7 from the current
document.
o Section 11.7.10 is replaced by Section 8.8 o Section 11.7.10 is replaced by Section 8.8 from the current
document.
o Sections 11.8, 11.8.1, 11.8.2, and 11.9, are unchanged. o Sections 11.8, 11.8.1, 11.8.2, and 11.9, are unchanged.
o Sections 11.10, 11.10.1, 11.10.2, and 11.10.3 are replaced by o Sections 11.10, 11.10.1, 11.10.2, and 11.10.3 are replaced by
Sections 12.2 through 12.2.3. Sections 12.2 through 12.2.3 from the current document.
o Section 11.11 is unchanged. o Section 11.11 is unchanged.
o New sections corresponding to Sections 9, 10, and 11 from the o New sections corresponding to Sections 9, 10, and 11 from the
current document appear next as additional sub-sections of current document appear next as additional sub-sections of
Section 11. Each of these has subsections, so there is a total Section 11. Each of these has subsections, so there is a total
of seventeen sections added. of seventeen sections added.
o Sections 12 through 14 are unchanged. o Sections 12 through 14 are unchanged.
skipping to change at page 97, line 43 skipping to change at page 101, line 11
* The description of NFS4ERR_MOVED in Section 15.1 is revised as * The description of NFS4ERR_MOVED in Section 15.1 is revised as
described in Section 13.3 of the current document. described in Section 13.3 of the current document.
* The description of the reclaim-related errors in section 15.1.9 * The description of the reclaim-related errors in section 15.1.9
is replaced by the revised descriptions in Section 13.7 of the is replaced by the revised descriptions in Section 13.7 of the
current document. current document.
o Sections 16 and 17 are unchanged. o Sections 16 and 17 are unchanged.
o Section 18 is unmodified except the o Section 18 is unmodified except for the following:
* Section 18.35 is replaced by Section 14 in the current * Section 18.35 is replaced by Section 14 in the current
document. document.
* Section 18.51 is replaced by Section 15 in the current * Section 18.51 is replaced by Section 15 in the current
document. document.
o Sections 19 through 23 are unchanged. o Sections 19 through 23 are unchanged.
In terms of top-level sections, exclusive of appendices: In terms of top-level sections, exclusive of appendices:
skipping to change at page 98, line 45 skipping to change at page 102, line 20
| Deleted | 0 | 1 | 4 | 0 | 4 | | Deleted | 0 | 1 | 4 | 0 | 4 |
| Modified | 5 | 3 | 0 | 2 | 2 | | Modified | 5 | 3 | 0 | 2 | 2 |
| Unchanged | 18 | 210 | 12 | 910 | 922 | | Unchanged | 18 | 210 | 12 | 910 | 922 |
| in RFC5661 | 23 | 220 | 37 | 927 | 964 | | in RFC5661 | 23 | 220 | 37 | 927 | 964 |
+------------+------+------+--------+------------+--------+ +------------+------+------+--------+------------+--------+
Acknowledgments Acknowledgments
The authors wish to acknowledge the important role of Andy Adamson of The authors wish to acknowledge the important role of Andy Adamson of
Netapp in clarifying the need for trunking discovery functionality, Netapp in clarifying the need for trunking discovery functionality,
and exploring the role of the location attributes in providing the and exploring the role of the file system location attributes in
necessary support. providing the necessary support.
The authors also wish to acknowledge the work of Xuan Qi of Oracle The authors also wish to acknowledge the work of Xuan Qi of Oracle
with NFSv4.1 client and server prototypes of transparent state with NFSv4.1 client and server prototypes of transparent state
migration functionality. migration functionality.
The authors wish to thank others that brought attention to important The authors wish to thank others that brought attention to important
issues. The comments of Trond Myklebust of Primary Data related to issues. The comments of Trond Myklebust of Primary Data related to
trunking helped to clarify the role of DNS in trunking discovery. trunking helped to clarify the role of DNS in trunking discovery.
Rick Macklem's comments brought attention to problems in the handling Rick Macklem's comments brought attention to problems in the handling
of the per-fs version of RECLAIM_COMPLETE. of the per-fs version of RECLAIM_COMPLETE.
 End of changes. 144 change blocks. 
422 lines changed or deleted 595 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/