draft-ietf-nfsv4-flex-files-11.txt   draft-ietf-nfsv4-flex-files-12.txt 
NFSv4 B. Halevy NFSv4 B. Halevy
Internet-Draft Internet-Draft
Intended status: Standards Track T. Haynes Intended status: Standards Track T. Haynes
Expires: January 19, 2018 Primary Data Expires: January 21, 2018 Primary Data
July 18, 2017 July 20, 2017
Parallel NFS (pNFS) Flexible File Layout Parallel NFS (pNFS) Flexible File Layout
draft-ietf-nfsv4-flex-files-11.txt draft-ietf-nfsv4-flex-files-12.txt
Abstract Abstract
The Parallel Network File System (pNFS) allows a separation between The Parallel Network File System (pNFS) allows a separation between
the metadata (onto a metadata server) and data (onto a storage the metadata (onto a metadata server) and data (onto a storage
device) for a file. The flexible file layout type is defined in this device) for a file. The flexible file layout type is defined in this
document as an extension to pNFS which allows the use of storage document as an extension to pNFS which allows the use of storage
devices in a fashion such that they require only a quite limited devices in a fashion such that they require only a quite limited
degree of interaction with the metadata server, using already degree of interaction with the metadata server, using already
existing protocols. Client side mirroring is also added to provide existing protocols. Client side mirroring is also added to provide
skipping to change at page 1, line 38 skipping to change at page 1, line 38
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 19, 2018. This Internet-Draft will expire on January 21, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 22 skipping to change at page 2, line 22
1.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 3
1.2. Difference Between a Data Server and a Storage Device . . 5 1.2. Difference Between a Data Server and a Storage Device . . 5
1.3. Requirements Language . . . . . . . . . . . . . . . . . . 6 1.3. Requirements Language . . . . . . . . . . . . . . . . . . 6
2. Coupling of Storage Devices . . . . . . . . . . . . . . . . . 6 2. Coupling of Storage Devices . . . . . . . . . . . . . . . . . 6
2.1. LAYOUTCOMMIT . . . . . . . . . . . . . . . . . . . . . . 6 2.1. LAYOUTCOMMIT . . . . . . . . . . . . . . . . . . . . . . 6
2.2. Fencing Clients from the Storage Device . . . . . . . . . 6 2.2. Fencing Clients from the Storage Device . . . . . . . . . 6
2.2.1. Implementation Notes for Synthetic uids/gids . . . . 7 2.2.1. Implementation Notes for Synthetic uids/gids . . . . 7
2.2.2. Example of using Synthetic uids/gids . . . . . . . . 8 2.2.2. Example of using Synthetic uids/gids . . . . . . . . 8
2.3. State and Locking Models . . . . . . . . . . . . . . . . 9 2.3. State and Locking Models . . . . . . . . . . . . . . . . 9
2.3.1. Loosely Coupled Locking Model . . . . . . . . . . . . 9 2.3.1. Loosely Coupled Locking Model . . . . . . . . . . . . 9
2.3.2. Tighly Coupled Locking Model . . . . . . . . . . . . 10 2.3.2. Tighly Coupled Locking Model . . . . . . . . . . . . 11
3. XDR Description of the Flexible File Layout Type . . . . . . 12 3. XDR Description of the Flexible File Layout Type . . . . . . 12
3.1. Code Components Licensing Notice . . . . . . . . . . . . 13 3.1. Code Components Licensing Notice . . . . . . . . . . . . 13
4. Device Addressing and Discovery . . . . . . . . . . . . . . . 14 4. Device Addressing and Discovery . . . . . . . . . . . . . . . 14
4.1. ff_device_addr4 . . . . . . . . . . . . . . . . . . . . . 14 4.1. ff_device_addr4 . . . . . . . . . . . . . . . . . . . . . 15
4.2. Storage Device Multipathing . . . . . . . . . . . . . . . 16 4.2. Storage Device Multipathing . . . . . . . . . . . . . . . 16
5. Flexible File Layout Type . . . . . . . . . . . . . . . . . . 17 5. Flexible File Layout Type . . . . . . . . . . . . . . . . . . 17
5.1. ff_layout4 . . . . . . . . . . . . . . . . . . . . . . . 17 5.1. ff_layout4 . . . . . . . . . . . . . . . . . . . . . . . 18
5.1.1. Error Codes from LAYOUTGET . . . . . . . . . . . . . 21 5.1.1. Error Codes from LAYOUTGET . . . . . . . . . . . . . 21
5.1.2. Client Interactions with FF_FLAGS_NO_IO_THRU_MDS . . 21 5.1.2. Client Interactions with FF_FLAGS_NO_IO_THRU_MDS . . 22
5.2. Interactions Between Devices and Layouts . . . . . . . . 22 5.2. Interactions Between Devices and Layouts . . . . . . . . 22
5.3. Handling Version Errors . . . . . . . . . . . . . . . . . 22 5.3. Handling Version Errors . . . . . . . . . . . . . . . . . 22
6. Striping via Sparse Mapping . . . . . . . . . . . . . . . . . 23 6. Striping via Sparse Mapping . . . . . . . . . . . . . . . . . 23
7. Recovering from Client I/O Errors . . . . . . . . . . . . . . 23 7. Recovering from Client I/O Errors . . . . . . . . . . . . . . 23
8. Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . 24 8. Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.1. Selecting a Mirror . . . . . . . . . . . . . . . . . . . 24 8.1. Selecting a Mirror . . . . . . . . . . . . . . . . . . . 25
8.2. Writing to Mirrors . . . . . . . . . . . . . . . . . . . 25 8.2. Writing to Mirrors . . . . . . . . . . . . . . . . . . . 25
8.2.1. Single Storage Device Updates Mirrors . . . . . . . . 25 8.2.1. Single Storage Device Updates Mirrors . . . . . . . . 25
8.2.2. Single Storage Device Updates Mirrors . . . . . . . . 25 8.2.2. Single Storage Device Updates Mirrors . . . . . . . . 26
8.2.3. Handling Write Errors . . . . . . . . . . . . . . . . 25 8.2.3. Handling Write Errors . . . . . . . . . . . . . . . . 26
8.2.4. Handling Write COMMITs . . . . . . . . . . . . . . . 26 8.2.4. Handling Write COMMITs . . . . . . . . . . . . . . . 27
8.3. Metadata Server Resilvering of the File . . . . . . . . . 27 8.3. Metadata Server Resilvering of the File . . . . . . . . . 27
9. Flexible Files Layout Type Return . . . . . . . . . . . . . . 27 9. Flexible Files Layout Type Return . . . . . . . . . . . . . . 27
9.1. I/O Error Reporting . . . . . . . . . . . . . . . . . . . 28 9.1. I/O Error Reporting . . . . . . . . . . . . . . . . . . . 29
9.1.1. ff_ioerr4 . . . . . . . . . . . . . . . . . . . . . . 28 9.1.1. ff_ioerr4 . . . . . . . . . . . . . . . . . . . . . . 29
9.2. Layout Usage Statistics . . . . . . . . . . . . . . . . . 29 9.2. Layout Usage Statistics . . . . . . . . . . . . . . . . . 29
9.2.1. ff_io_latency4 . . . . . . . . . . . . . . . . . . . 29 9.2.1. ff_io_latency4 . . . . . . . . . . . . . . . . . . . 30
9.2.2. ff_layoutupdate4 . . . . . . . . . . . . . . . . . . 30 9.2.2. ff_layoutupdate4 . . . . . . . . . . . . . . . . . . 30
9.2.3. ff_iostats4 . . . . . . . . . . . . . . . . . . . . . 30 9.2.3. ff_iostats4 . . . . . . . . . . . . . . . . . . . . . 31
9.3. ff_layoutreturn4 . . . . . . . . . . . . . . . . . . . . 32 9.3. ff_layoutreturn4 . . . . . . . . . . . . . . . . . . . . 32
10. Flexible Files Layout Type LAYOUTERROR . . . . . . . . . . . 32 10. Flexible Files Layout Type LAYOUTERROR . . . . . . . . . . . 33
11. Flexible Files Layout Type LAYOUTSTATS . . . . . . . . . . . 32 11. Flexible Files Layout Type LAYOUTSTATS . . . . . . . . . . . 33
12. Flexible File Layout Type Creation Hint . . . . . . . . . . . 33 12. Flexible File Layout Type Creation Hint . . . . . . . . . . . 33
12.1. ff_layouthint4 . . . . . . . . . . . . . . . . . . . . . 33 12.1. ff_layouthint4 . . . . . . . . . . . . . . . . . . . . . 34
13. Recalling a Layout . . . . . . . . . . . . . . . . . . . . . 34 13. Recalling a Layout . . . . . . . . . . . . . . . . . . . . . 34
13.1. CB_RECALL_ANY . . . . . . . . . . . . . . . . . . . . . 34 13.1. CB_RECALL_ANY . . . . . . . . . . . . . . . . . . . . . 34
14. Client Fencing . . . . . . . . . . . . . . . . . . . . . . . 35 14. Client Fencing . . . . . . . . . . . . . . . . . . . . . . . 35
15. Security Considerations . . . . . . . . . . . . . . . . . . . 35 15. Security Considerations . . . . . . . . . . . . . . . . . . . 36
15.1. Kerberized File Access . . . . . . . . . . . . . . . . . 36 15.1. Kerberized File Access . . . . . . . . . . . . . . . . . 37
15.1.1. Loosely Coupled . . . . . . . . . . . . . . . . . . 36 15.1.1. Loosely Coupled . . . . . . . . . . . . . . . . . . 37
15.1.2. Tightly Coupled . . . . . . . . . . . . . . . . . . 36 15.1.2. Tightly Coupled . . . . . . . . . . . . . . . . . . 37
16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 37 16. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 37
17. References . . . . . . . . . . . . . . . . . . . . . . . . . 38 17. References . . . . . . . . . . . . . . . . . . . . . . . . . 38
17.1. Normative References . . . . . . . . . . . . . . . . . . 38 17.1. Normative References . . . . . . . . . . . . . . . . . . 38
17.2. Informative References . . . . . . . . . . . . . . . . . 38 17.2. Informative References . . . . . . . . . . . . . . . . . 39
Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 39 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 39
Appendix B. RFC Editor Notes . . . . . . . . . . . . . . . . . . 39 Appendix B. RFC Editor Notes . . . . . . . . . . . . . . . . . . 40
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 39 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40
1. Introduction 1. Introduction
In the parallel Network File System (pNFS), the metadata server In the parallel Network File System (pNFS), the metadata server
returns layout type structures that describe where file data is returns layout type structures that describe where file data is
located. There are different layout types for different storage located. There are different layout types for different storage
systems and methods of arranging data on storage devices. This systems and methods of arranging data on storage devices. This
document defines the flexible file layout type used with file-based document defines the flexible file layout type used with file-based
data servers that are accessed using the Network File System (NFS) data servers that are accessed using the Network File System (NFS)
protocols: NFSv3 [RFC1813], NFSv4.0 [RFC7530], NFSv4.1 [RFC5661], and protocols: NFSv3 [RFC1813], NFSv4.0 [RFC7530], NFSv4.1 [RFC5661], and
skipping to change at page 6, line 27 skipping to change at page 6, line 27
The coupling of the metadata server with the storage devices can be The coupling of the metadata server with the storage devices can be
either tight or loose. In a tight coupling, there is a control either tight or loose. In a tight coupling, there is a control
protocol present to manage security, LAYOUTCOMMITs, etc. With a protocol present to manage security, LAYOUTCOMMITs, etc. With a
loose coupling, the only control protocol might be a version of NFS. loose coupling, the only control protocol might be a version of NFS.
As such, semantics for managing security, state, and locking models As such, semantics for managing security, state, and locking models
MUST be defined. MUST be defined.
2.1. LAYOUTCOMMIT 2.1. LAYOUTCOMMIT
The metadata server has the responsibility, upon receiving a Regardless of the coupling model, the metadata server has the
LAYOUTCOMMIT (see Section 18.42 of [RFC5661]), of ensuring that the responsibility, upon receiving a LAYOUTCOMMIT (see Section 18.42 of
semantics of pNFS are respected (see Section 12.5.4 of [RFC5661]). [RFC5661]), of ensuring that the semantics of pNFS are respected (see
These do include a requirement that data written to data storage Section 12.5.4 of [RFC5661]). These do include a requirement that
device be stable before the occurance of the LAYOUTCOMMIT. data written to data storage device be stable before the occurrence
of the LAYOUTCOMMIT.
It is the responsibility of the client to make sure the data file is It is the responsibility of the client to make sure the data file is
stable before the metadata server begins to query the storage devices stable before the metadata server begins to query the storage devices
about the changes to the file. If any WRITE to a storage device did about the changes to the file. If any WRITE to a storage device did
not result with stable_how equal to FILE_SYNC, a LAYOUTCOMMIT to the not result with stable_how equal to FILE_SYNC, a LAYOUTCOMMIT to the
metadata server MUST be preceded by a COMMIT to the storage devices metadata server MUST be preceded by a COMMIT to the storage devices
written to. Note that if the client has not done a COMMIT to the written to. Note that if the client has not done a COMMIT to the
storage device, then the LAYOUTCOMMIT might not be synchronized to storage device, then the LAYOUTCOMMIT might not be synchronized to
the last WRITE operation to the storage device. the last WRITE operation to the storage device.
skipping to change at page 7, line 5 skipping to change at page 7, line 6
With loosely coupled storage devices, the metadata server uses With loosely coupled storage devices, the metadata server uses
synthetic uids and gids for the data file, where the uid owner of the synthetic uids and gids for the data file, where the uid owner of the
data file is allowed read/write access and the gid owner is allowed data file is allowed read/write access and the gid owner is allowed
read only access. As part of the layout (see ffds_user and read only access. As part of the layout (see ffds_user and
ffds_group in Section 5.1), the client is provided with the user and ffds_group in Section 5.1), the client is provided with the user and
group to be used in the Remote Procedure Call (RPC) [RFC5531] group to be used in the Remote Procedure Call (RPC) [RFC5531]
credentials needed to access the data file. Fencing off of clients credentials needed to access the data file. Fencing off of clients
is achieved by the metadata server changing the synthetic uid and/or is achieved by the metadata server changing the synthetic uid and/or
gid owners of the data file on the storage device to implicitly gid owners of the data file on the storage device to implicitly
revoke the outstanding RPC credentials. revoke the outstanding RPC credentials. A client presenting the
wrong credential for the deisred access will get a NFS4ERR_ACCESS
error.
With this loosely coupled model, the metadata server is not able to With this loosely coupled model, the metadata server is not able to
fence off a single client, it is forced to fence off all clients. fence off a single client, it is forced to fence off all clients.
However, as the other clients react to the fencing, returning their However, as the other clients react to the fencing, returning their
layouts and trying to get new ones, the metadata server can hand out layouts and trying to get new ones, the metadata server can hand out
a new uid and gid to allow access. a new uid and gid to allow access.
Note: it is recommended to implement common access control methods at Note: it is recommended to implement common access control methods at
the storage device filesystem to allow only the metadata server root the storage device filesystem to allow only the metadata server root
(super user) access to the storage device, and to set the owner of (super user) access to the storage device, and to set the owner of
skipping to change at page 7, line 31 skipping to change at page 7, line 34
With tightly coupled storage devices, the metadata server sets the With tightly coupled storage devices, the metadata server sets the
user and group owners, mode bits, and ACL of the data file to be the user and group owners, mode bits, and ACL of the data file to be the
same as the metadata file. And the client must authenticate with the same as the metadata file. And the client must authenticate with the
storage device and go through the same authorization process it would storage device and go through the same authorization process it would
go through via the metadata server. In the case of tight coupling, go through via the metadata server. In the case of tight coupling,
fencing is the responsibility of the control protocol and is not fencing is the responsibility of the control protocol and is not
described in detail here. However, implementations of the tight described in detail here. However, implementations of the tight
coupling locking model (see Section 2.3), will need a way to prevent coupling locking model (see Section 2.3), will need a way to prevent
access by certain clients to specific files by invalidating the access by certain clients to specific files by invalidating the
corresponding stateids on the storage device. corresponding stateids on the storage device. In such a scenario,
the client will be given an error of NFS4ERR_BAD_STATEID.
The client need not know the model used between the metadata server
and the storage device. It need only react consistently to any
errors in interacting with the storage device. It should both return
the layout and error to the metadata server and ask for a new layout.
At that point, the metadata server can either hand out a new layout,
hand out no layout (forcing the I/O through it), or deny the client
further access to the file.
2.2.1. Implementation Notes for Synthetic uids/gids 2.2.1. Implementation Notes for Synthetic uids/gids
The selection method for the synthetic uids and gids to be used for The selection method for the synthetic uids and gids to be used for
fencing in loosely coupled storage devices is strictly an fencing in loosely coupled storage devices is strictly an
implementation issue. I.e., an administrator might restrict a range implementation issue. I.e., an administrator might restrict a range
of such ids available to the Lightweight Directory Access Protocol of such ids available to the Lightweight Directory Access Protocol
(LDAP) 'uid' field [RFC4519]. She might also be able to choose an id (LDAP) 'uid' field [RFC4519]. She might also be able to choose an id
that would never be used to grant acccess. Then when the metadata that would never be used to grant acccess. Then when the metadata
server had a request to access a file, a SETATTR would be sent to the server had a request to access a file, a SETATTR would be sent to the
skipping to change at page 9, line 27 skipping to change at page 9, line 39
always treated as loosely coupled. always treated as loosely coupled.
o NFSv4.1+ storage devices that do not return the o NFSv4.1+ storage devices that do not return the
EXCHGID4_FLAG_USE_PNFS_DS flag set to EXCHANGE_ID are indicating EXCHGID4_FLAG_USE_PNFS_DS flag set to EXCHANGE_ID are indicating
that they are to be treated as loosely coupled. From the locking that they are to be treated as loosely coupled. From the locking
viewpoint they are treated in the same way as NFSv4.0 storage viewpoint they are treated in the same way as NFSv4.0 storage
devices. devices.
o NFSv4.1+ storage devices that do identify themselves with the o NFSv4.1+ storage devices that do identify themselves with the
EXCHGID4_FLAG_USE_PNFS_DS flag set to EXCHANGE_ID are considered EXCHGID4_FLAG_USE_PNFS_DS flag set to EXCHANGE_ID are considered
strongly coupled. They would use a back-end control protocol to tightly coupled. They would use a back-end control protocol to
implement the global stateid model as described in [RFC5661]. implement the global stateid model as described in [RFC5661].
2.3.1. Loosely Coupled Locking Model 2.3.1. Loosely Coupled Locking Model
When locking-related operations are requested, they are primarily When locking-related operations are requested, they are primarily
dealt with by the metadata server, which generates the appropriate dealt with by the metadata server, which generates the appropriate
stateids. When an NFSv4 version is used as the data access protocol, stateids. When an NFSv4 version is used as the data access protocol,
the metadata server may make stateid-related requests of the storage the metadata server may make stateid-related requests of the storage
devices. However, it is not required to do so and the resulting devices. However, it is not required to do so and the resulting
stateids are known only to the metadata server and the storage stateids are known only to the metadata server and the storage
skipping to change at page 20, line 12 skipping to change at page 20, line 30
Section 5.3 for how to handle versioning issues between the client Section 5.3 for how to handle versioning issues between the client
and storage devices. and storage devices.
For tight coupling, ffds_stateid provides the stateid to be used by For tight coupling, ffds_stateid provides the stateid to be used by
the client to access the file. For loose coupling and a NFSv4 the client to access the file. For loose coupling and a NFSv4
storage device, the client may use an anonymous stateid to perform I/ storage device, the client may use an anonymous stateid to perform I/
O on the storage device as there is no use for the metadata server O on the storage device as there is no use for the metadata server
stateid (no control protocol). In such a scenario, the server MUST stateid (no control protocol). In such a scenario, the server MUST
set the ffds_stateid to be the anonymous stateid. set the ffds_stateid to be the anonymous stateid.
This specification of the ffds_stateid is mostly broken for the This specification of the ffds_stateid restricts both models for
tightly coupled model. There needs to exist a one to one mapping NFSv4.x storage protocols:
from ffds_stateid to ffds_fh_vers - each open file on the storage
device might need an open stateid. As there are established loosely
coupled implementations of this version of the protocol, the only
viable approaches for a tightly coupled implementation would be to
either use an anonymous stateid for the ffds_stateid or restrict the
size of the ffds_fh_vers to be one. Fixing this issue will require a
new version of the protocol.
[[AI14: One reviewer points out for loosely coupled, we can use the loosely couple: the stateid has to be an anonymous stateid,
anon stateid and for tightly coupled we can use the "global stateid".
These make it appear that the bug in the spec was actually a feature. tightly couple: the stateid has to be a global stateid.
The intent here is to own up to the bug and shipping code. Can it be
said nicer? --TH]] These stem from a mismatch of ffds_stateid being a singleton and
ffds_fh_vers being an array - each open file on the storage device
might need an open stateid. As there are established loosely coupled
implementations of this version of the protocol, it can not be fixed.
If an implementation needs a different statedid per file handle, then
this issue will require a new version of the protocol.
For loosely coupled storage devices, ffds_user and ffds_group provide For loosely coupled storage devices, ffds_user and ffds_group provide
the synthetic user and group to be used in the RPC credentials that the synthetic user and group to be used in the RPC credentials that
the client presents to the storage device to access the data files. the client presents to the storage device to access the data files.
For tightly coupled storage devices, the user and group on the For tightly coupled storage devices, the user and group on the
storage device will be the same as on the metadata server. I.e., if storage device will be the same as on the metadata server. I.e., if
ffdv_tightly_coupled (see Section 4.1) is set, then the client MUST ffdv_tightly_coupled (see Section 4.1) is set, then the client MUST
ignore both ffds_user and ffds_group. ignore both ffds_user and ffds_group.
The allowed values for both ffds_user and ffds_group are specified in The allowed values for both ffds_user and ffds_group are specified in
skipping to change at page 21, line 13 skipping to change at page 21, line 30
mirror to access is discussed in Section 8.1. mirror to access is discussed in Section 8.1.
ffl_flags is a bitmap that allows the metadata server to inform the ffl_flags is a bitmap that allows the metadata server to inform the
client of particular conditions that may result from the more or less client of particular conditions that may result from the more or less
tight coupling of the storage devices. tight coupling of the storage devices.
FF_FLAGS_NO_LAYOUTCOMMIT: can be set to indicate that the client is FF_FLAGS_NO_LAYOUTCOMMIT: can be set to indicate that the client is
not required to send LAYOUTCOMMIT to the metadata server. not required to send LAYOUTCOMMIT to the metadata server.
F_FLAGS_NO_IO_THRU_MDS: can be set to indicate that the client F_FLAGS_NO_IO_THRU_MDS: can be set to indicate that the client
SHOULD not send I/O operations to the metadata server. I.e., even should not send I/O operations to the metadata server. I.e., even
if the client could determine that there was a network diconnect if the client could determine that there was a network diconnect
to a storage device, the client SHOULD not try to proxy the I/O to a storage device, the client should not try to proxy the I/O
through the metadata server. through the metadata server.
FF_FLAGS_NO_READ_IO: can be set to indicate that the client SHOULD FF_FLAGS_NO_READ_IO: can be set to indicate that the client should
not send READ requests with the layouts of iomode not send READ requests with the layouts of iomode
LAYOUTIOMODE4_RW. Instead, it should request a layout of iomode LAYOUTIOMODE4_RW. Instead, it should request a layout of iomode
LAYOUTIOMODE4_READ from the metadata server. LAYOUTIOMODE4_READ from the metadata server.
FF_FLAGS_WRITE_ONE_MIRROR: can be set to indicate that the client FF_FLAGS_WRITE_ONE_MIRROR: can be set to indicate that the client
only needs to update one of the mirrors (see Section 8.2). only needs to update one of the mirrors (see Section 8.2).
5.1.1. Error Codes from LAYOUTGET 5.1.1. Error Codes from LAYOUTGET
[RFC5661] provides little guidance as to how the client is to proceed [RFC5661] provides little guidance as to how the client is to proceed
skipping to change at page 28, line 27 skipping to change at page 28, line 46
layoutreturn4 lora_layoutreturn; layoutreturn4 lora_layoutreturn;
}; };
<CODE ENDS> <CODE ENDS>
If the lora_layout_type layout type is LAYOUT4_FLEX_FILES and the If the lora_layout_type layout type is LAYOUT4_FLEX_FILES and the
lr_returntype is LAYOUTRETURN4_FILE, then the lrf_body opaque value lr_returntype is LAYOUTRETURN4_FILE, then the lrf_body opaque value
is defined by ff_layoutreturn4 (See Section 9.3). It allows the is defined by ff_layoutreturn4 (See Section 9.3). It allows the
client to report I/O error information or layout usage statistics client to report I/O error information or layout usage statistics
back to the metadata server as defined below. Note that while the back to the metadata server as defined below. Note that while the
data strucures are built on concepts introduced in NFSv4.2, the data structures are built on concepts introduced in NFSv4.2, the
effective discriminated union (lora_layout_type combined with effective discriminated union (lora_layout_type combined with
ff_layoutreturn4) allows for a NFSv4.1 metadata server to utilize the ff_layoutreturn4) allows for a NFSv4.1 metadata server to utilize the
data. data.
9.1. I/O Error Reporting 9.1. I/O Error Reporting
9.1.1. ff_ioerr4 9.1.1. ff_ioerr4
<CODE BEGINS> <CODE BEGINS>
skipping to change at page 36, line 29 skipping to change at page 37, line 9
question. It is REQUIRED that this be done before committing to the question. It is REQUIRED that this be done before committing to the
new permissions and/or ACL. By requesting new layouts, the clients new permissions and/or ACL. By requesting new layouts, the clients
will reauthorize access against the modified access control metadata. will reauthorize access against the modified access control metadata.
Recalling the layouts in this case is intended to prevent clients Recalling the layouts in this case is intended to prevent clients
from getting an error on I/Os done after the client was fenced off. from getting an error on I/Os done after the client was fenced off.
15.1. Kerberized File Access 15.1. Kerberized File Access
15.1.1. Loosely Coupled 15.1.1. Loosely Coupled
RPCSEC_GSS version 3 (RPCSEC_GSSv3) [rpcsec_gssv3] could be used to RPCSEC_GSS version 3 (RPCSEC_GSSv3) [RFC7861] could be used to
authorize the client to the storage device on behalf of the metadata authorize the client to the storage device on behalf of the metadata
server. This would require that each of the metadata server, storage server. This would require that each of the metadata server, storage
device, and client would have to implement RPCSEC_GSSv3. The second device, and client would have to implement RPCSEC_GSSv3. The second
requirement does not match the intent of the loosely coupled model requirement does not match the intent of the loosely coupled model
that the storage device need not be modified. that the storage device need not be modified.
Under this coupling model, the principal used to authenticate the Under this coupling model, the principal used to authenticate the
metadata file is different than that used to authenticate the data metadata file is different than that used to authenticate the data
file. For the metadata server, the user credentials would be file. For the metadata server, the user credentials would be
generated by the same Kerberos server as the client. However, for generated by the same Kerberos server as the client. However, for
skipping to change at page 37, line 12 skipping to change at page 37, line 39
there are no security issues related to using Kerberos with a tightly there are no security issues related to using Kerberos with a tightly
coupled system. coupled system.
16. IANA Considerations 16. IANA Considerations
[RFC5661] introduced a registry for "pNFS Layout Types Registry" and [RFC5661] introduced a registry for "pNFS Layout Types Registry" and
as such, new layout type numbers need to be assigned by IANA. This as such, new layout type numbers need to be assigned by IANA. This
document defines the protocol associated with the existing layout document defines the protocol associated with the existing layout
type number, LAYOUT4_FLEX_FILES (see Table 1). type number, LAYOUT4_FLEX_FILES (see Table 1).
+--------------------+-------+--------+-----+----------------+ +--------------------+-------+----------+-----+----------------+
| Layout Type Name | Value | RFC | How | Minor Versions | | Layout Type Name | Value | RFC | How | Minor Versions |
+--------------------+-------+--------+-----+----------------+ +--------------------+-------+----------+-----+----------------+
| LAYOUT4_FLEX_FILES | 0x4 | RFCTDB | L | 1 | | LAYOUT4_FLEX_FILES | 0x4 | RFCTBD10 | L | 1 |
+--------------------+-------+--------+-----+----------------+ +--------------------+-------+----------+-----+----------------+
Table 1: Layout Type Assignments Table 1: Layout Type Assignments
[RFC5661] also introduced a registry called "NFSv4 Recallable Object [RFC5661] also introduced a registry called "NFSv4 Recallable Object
Types Registry". This document defines new recallable objects for Types Registry". This document defines new recallable objects for
RCA4_TYPE_MASK_FF_LAYOUT_MIN and RCA4_TYPE_MASK_FF_LAYOUT_MAX (see RCA4_TYPE_MASK_FF_LAYOUT_MIN and RCA4_TYPE_MASK_FF_LAYOUT_MAX (see
Table 2). Table 2).
+------------------------------+-------+--------+-----+-------------+ +------------------------------+-------+----------+-----+-----------+
| Recallable Object Type Name | Value | RFC | How | Minor | | Recallable Object Type Name | Value | RFC | How | Minor |
| | | | | Versions | | | | | | Versions |
+------------------------------+-------+--------+-----+-------------+ +------------------------------+-------+----------+-----+-----------+
| RCA4_TYPE_MASK_FF_LAYOUT_MIN | 16 | RFCTDB | L | 1 | | RCA4_TYPE_MASK_FF_LAYOUT_MIN | 16 | RFCTBD10 | L | 1 |
| RCA4_TYPE_MASK_FF_LAYOUT_MAX | 17 | RFCTDB | L | 1 | | RCA4_TYPE_MASK_FF_LAYOUT_MAX | 17 | RFCTBD10 | L | 1 |
+------------------------------+-------+--------+-----+-------------+ +------------------------------+-------+----------+-----+-----------+
Table 2: Recallable Object Type Assignments Table 2: Recallable Object Type Assignments
Note, [RFC5661] should have also defined (see Table 3): Note, [RFC5661] should have also defined (see Table 3):
+-------------------------------+------+-----------+-----+----------+ +-------------------------------+------+-----------+-----+----------+
| Recallable Object Type Name | Valu | RFC | How | Minor | | Recallable Object Type Name | Valu | RFC | How | Minor |
| | e | | | Versions | | | e | | | Versions |
+-------------------------------+------+-----------+-----+----------+ +-------------------------------+------+-----------+-----+----------+
| RCA4_TYPE_MASK_OTHER_LAYOUT_M | 12 | [RFC5661] | L | 1 | | RCA4_TYPE_MASK_OTHER_LAYOUT_M | 12 | [RFC5661] | L | 1 |
skipping to change at page 39, line 5 skipping to change at page 39, line 28
ietf-nfsv4-layout-types-04 (Work In Progress), January ietf-nfsv4-layout-types-04 (Work In Progress), January
2016. 2016.
17.2. Informative References 17.2. Informative References
[RFC4519] Sciberras, A., Ed., "Lightweight Directory Access Protocol [RFC4519] Sciberras, A., Ed., "Lightweight Directory Access Protocol
(LDAP): Schema for User Applications", RFC 4519, DOI (LDAP): Schema for User Applications", RFC 4519, DOI
10.17487/RFC4519, June 2006, 10.17487/RFC4519, June 2006,
<http://www.rfc-editor.org/info/rfc4519>. <http://www.rfc-editor.org/info/rfc4519>.
[rpcsec_gssv3] [RFC7861] Adamson, W. and N. Williams, "Remote Procedure Call (RPC)
Adamson, W. and N. Williams, "Remote Procedure Call (RPC) Security Version 3", November 2016.
Security Version 3", November 2014.
Appendix A. Acknowledgments Appendix A. Acknowledgments
Those who provided miscellaneous comments to early drafts of this Those who provided miscellaneous comments to early drafts of this
document include: Matt W. Benjamin, Adam Emerson, J. Bruce Fields, document include: Matt W. Benjamin, Adam Emerson, J. Bruce Fields,
and Lev Solomonov. and Lev Solomonov.
Those who provided miscellaneous comments to the final drafts of this Those who provided miscellaneous comments to the final drafts of this
document include: Anand Ganesh, Robert Wipfel, Gobikrishnan document include: Anand Ganesh, Robert Wipfel, Gobikrishnan
Sundharraj, Trond Myklebust, and Rick Macklem. Sundharraj, Trond Myklebust, and Rick Macklem.
 End of changes. 31 change blocks. 
68 lines changed or deleted 77 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/