NFSv4                                                          T. Haynes
Internet-Draft                                              Primary Data
Updates: 5661 (if approved)                             January 28, 2016                                July 20, 2017
Intended status: Standards Track
Expires: July 31, 2016 January 21, 2018

                   Requirements for pNFS Layout Types


   This document provides help in distinguishing between defines the requirements for Network File System (NFS) version 4.1's Parallel which individual pNFS layout
   types need to meet in order to work within the parallel NFS (pNFS)
   framework as defined in RFC5661.  In so doing, it aims to more
   clearly distinguish between requirements for pNFS as a whole and
   those those specifically directed to the pNFS File Layout.  The lack
   of a clear separation between the two set of requirements has been
   troublesome for those specifying and evaluating new Layout Types.  As  In
   this regard, this document clarifies RFC5661, it effectively updates RFC5661.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on July 31, 2016. January 21, 2018.

Copyright Notice

   Copyright (c) 2016 2017 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   ( in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Definitions . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Difference Between a Data Server  Use of the Terms "Data Server" and a Storage Device "Storage Device" . .   4 .   5
     2.2.  Requirements Language . . . . . . . . . . . . . . . . . .   4   6
   3.  The Control Protocol  . . . . . . . . . . . . . . . . . . . .   4   6
     3.1.  Protocol Requirements REQUIREMENTS . . . . . . . . . . . . . . . . . .   5   7
     3.2.  Non-protocol Requirements . . . . .  Undocumented Protocol REQUIREMENTS  . . . . . . . . . . .   6   9
     3.3.  Editorial Requirements  . . . . . . . . . . . . . . . . .   6  10
   4.  Implementations in  Specifications of Existing Layout Types . . . . . . . . . .   7 .  10
     4.1.  File Layout Type  . . . . . . . . . . . . . . . . . . . .   7  10
     4.2.  Block Layout Type . . . . . . . . . . . . . . . . . . . .   7  12
     4.3.  Object Layout Type  . . . . . . . . . . . . . . . . . . .   8  13
   5.  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .   9  13
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .   9  14
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10  14
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10  15
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .  10  15
     8.2.  Informative References  . . . . . . . . . . . . . . . . .  10  15
   Appendix A.  Acknowledgments  . . . . . . . . . . . . . . . . . .  10  15
   Appendix B.  RFC Editor Notes . . . . . . . . . . . . . . . . . .  10  15
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  11  15

1.  Introduction


   The concept of layout type has a central role in the definition and
   implementation of Parallel Network File System (pNFS) (pNFS).  Clients and
   servers implementing different layout types behave differently in
   many ways while conforming to the File overall pNFS framework defined in
   [RFC5661] and this document.  Layout Type
   were types may differ in:

   o  The method used to do I/O operations directed to data storage

   o  The requirements for communication between the metadata server
      (MDS) and the storage devices.

   o  The means used to ensure that I/O requests are only processed when
      the client holds an appropriate layout.

   o  The format and interpretation of nominally opaque data fields in
      pNFS-related NFSv4.x data structures.

   Such matters are defined in a standards-track layout type
   specification.  Except for the files layout type, which was defined
   in Section 13 of [RFC5661], existing layout types are defined in
   their own standards-track documents and it is anticipated that new
   layout type will be defined in similar documents.

   The file layout type was defined in the Network File System (NFS)
   version 4.1 protocol
   specification, specification [RFC5661].  The Block Layout Type block layout type
   was defined in [RFC5663] and the Object Layout Type object layout type was in turn
   defined in [RFC5664].

   Some implementers have interpreted the text in Sections 12 ("Parallel
   NFS (pNFS)") and 13 ("NFSv4.1 as a Storage Protocol in pNFS: the File
   Layout Type") of [RFC5661] as both being strictly for applying only to the File Layout
   Type.  I.e., since file
   layout type.  Because Section 13 was not covered in a separate RFC
   standards-track document like those for both the Block block and Object Layout Types, object
   layout types, there is had been some confusion as to the
   responsibilities of both the Metadata Server
   (MDS) metadata server and the Data Servers data servers
   (DS) which were laid out in Section 12.

   As a consequence, new internet drafts (see [FlexFiles] and [Lustre])
   may struggle to meet the requirements to be a pNFS Layout Type. layout type.  This
   document clarifies what are specifies the Layout Type layout type independent requirements placed on
   all Layout Types, layout types, whether one of the original three or any new

2.  Definitions

   control protocol:  is a set of requirements communication requirements:  define for a layout type the communication of
      details regarding information on layouts, stateids, file metadata,
      and file data which must be communicated between the metadata
      server and the storage devices.

   control protocol:  defines a particular mechanism that an
      implementation of a layout type would use to meet the control
      communication requirement for that layout type.  This need not be
      a protocol as normally understood.  In some cases the same
      protocol my be used as a control protocol and data access

   (file) data:  is that part of the file system object which describes contains
      the data to read or writen.  It is the contents of the payload object and
      not the object.  E.g., it is attributes of the file contents.

   Data Server object.

   data server (DS):  is one of the a pNFS servers server which provide provides the file's data
      when the
      contents of a file system object which is accessed over a regular file. file-based
      protocol.  Note that this usage differs from that in [RFC5661]
      which applies the term in some cases even when other sorts of
      protocols are being used.  Depending on the layout, there might be
      one or more data servers over which the data is striped.  Note that while  While
      the metadata server is strictly accessed over the NFSv4.1
      protocol, depending on the Layout Type, layout type, the data server could be
      accessed via any file access protocol that meets the pNFS

      See Section 2.1 for a comparison of this term and "data storage

   fencing:  is when the process by which the metadata server prevents the
      storage devices from processing I/O from a specific client to a
      specific file.

   layout:  informs  contains information a client uses to access file data on a
      storage device.  This information will include specification of which
      the protocol (layout type) and the identity of the storage devices it needs to
      communicate with (and over which protocol)
      to perform I/O on a
      file. be used.

      The layout might also provide some hints about how bulk of the
      storage is physically organized.

   layout iomode:  describes whether contents of the layout granted to the client is
      for read or read/write I/O. are defined in [RFC5661]
      as nominally opaque, but individual layout types may specify their
      own interpretation of layout data.

   layout iomode:  see Section 1.

   layout stateid:  is a 128-bit quantity returned by a server that
      uniquely defines the layout state provided by the server for a
      specific layout that describes a Layout Type layout type and file (see
      Section 12.5.2 of [RFC5661]).  Further, Section 12.5.3 describes
      the difference
      differences in handling between a layout stateid stateids and a normal stateid.

   Layout Type: other stateid

   layout type:  describes both the storage protocol used to access the
      data and the aggregation scheme used to lays lay out the file data on
      the underlying storage devices.

   loose coupling:  describes when the control protocol, between a
      metadata server and storage device, is a storage protocol.

   (file) metadata:  is that part of the file system object which
      describes that
      contains various descriptive data relevant to the object and not file object, as
      opposed to the payload.  E.g., it file data itself.  This could be include the time since of
      last modification, access, access time, eof position, etc.

   Metadata Server

   metadata server (MDS):  is the pNFS server which provides metadata
      information for a file system object.  It also is responsible for
      generating, recalling, and revoking layouts for file system objects.  Note that the MDS is
      objects, for performing directory operations, and for directory-based operations. performing I
      /O operations to regular files when the clients direct these to
      the metadata server itself.

   recalling a layout:  is  occurs when the metadata server uses issues a back channel callbck
      to inform the client that the layout is to be returned in a
      graceful manner.  Note that the client could be able to flush any
      writes, etc., before replying to the metadata server.

   revoking a layout:  is  occurs when the metadata server invalidates the a
      specific layout such that neither Once revocation occurs, the metadata server nor will
      not accept as valid any reference to the revoked layout and a
      storage device will not accept any client access from based on the client with that

   stateid:  is a 128-bit quantity returned by a server that uniquely
      defines the open and locking states set of locking-related state provided by the server for a
      specific open-owner server.
      Stateids may designate state related to open files, to byte-range
      locks, to delegations, or lock-owner/open-owner pair for a specific
      file and type of lock. to layouts.

   storage device:  designates the target to which clients may direct I/
      O requests when they hold an appropriate layout.  Note that each
      data server is another term used almost interchangeably with a storage device but that some storage device are
      not data server. servers.  See Section 2.1 for further discussion.

   storage protocol:  is the nuances between the two.

2.1.  Difference Between a Data Server and a Storage Device

   We defined a data server as a pNFS server, which implies that it can
   utilize the NFSv4.1 protocol used by clients to do I/O
      operations to communicate with the client.  As
   such, only the File Layout Type would currently meet this
   requirement.  The more generic concept is a storage device, which can
   use any protocol to communicate with the client.  The requirements
   for a Each layout type may specify its
      own storage device to act together with the metadata server to
   provide data to a client are that there protocol.  It is a Layout Type
   specification possible for the given protocol and that the metadata server has
   granted a layout type to the client.  Note that nothing precludes there
   being specify
      multiple supported Layout Types (i.e., protocols) access protocols.

   tight coupling:  describes when the control protocol, between a
      metadata server, storage devices, server and client.

   As storage device device, is either a propritary
      approach or based on a standards-track document.

2.1.  Use of the more encompassing terminology, Terms "Data Server" and "Storage Device"

   In [RFC5661], these the two terms of "Data Server" and "Storage
   Device" are used somewhat inconsistently:

   o  In chapter 12, where pNFS in general is discussed, the term
      "storage device" is used.

   o  In chapter 13, where the file layout type is discussed, the term
      "data server" is used.

   o  In other chapters, the term "data server" is used, even in
      contexts where the storage access type is not NFSv4.1 or any other
      file access protocol.

   As this document
   utilizes deals with pNFS in general, it over uses the more generic
   term "storage device" in preference to "data server".  The term "data
   server" is used only in contexts in which a file server is used as a
   storage device.  Note that every data server. server is a storage device but
   that storage devices which use protocols which are not file access
   protocol are not data servers.

   Since a given storage device may support multiple layout types, a
   given device can potentially act as a data server for some set of
   storage protocols while simultaneously acting as a non-data-server
   storage device for others.

2.2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   document are to be interpreted as described in [RFC2119].

3.  The Control Protocol

   In Section 12.2.6 of [RFC5661], the control protocol is was introduced.
   There have been no published specifications for control protocols, and indeed
   there need not be such a protocol in use for any given
   implementation. protocols as
   yet.  The control protocol is actually a set of denotes any mechanism used to meet the
   requirements provided that apply to describe the interaction between the metadata
   server and the storage device.  When specifying device such that they present a new Layout
   Type, consistent
   interface to the client.  Particular implementations may satisfy this
   requirement in any manner they choose and the mechanism chosen may
   not be described as a protocol.  Specifications defining document MUST layout types
   need to clearly show how it meets these
   requirements, implementations can meet the requirements
   discussed below, especially with respect to the those that have security

3.1.  Protocol Requirements

   The broad  In addition, such specifications may find it necessary
   to impose requirements on implementations of such interactions between the metadata
   server and layout type to
   ensure appropriate interoperability.

   In some cases, there may be no control protocol other than the
   storage devices are:

   (1)  NFSv4.1 protocol.  This is often described as using a "loose
   coupling" model.  In such cases, the assumption is that the metadata
   server, storage devices, and client may be changed independently and
   that the implementation requirements in the layout type specification
   need to ensure this degree of interoperability.  This model is used
   in the block and object layout type specification.

   In some cases, there may be no control protocol other than the
   storage In other cases, it is assumed that there may be purpose-built
   control protocol which may be different for different implementations
   of the metadata server and data server.  In such cases, the
   assumption is that the metadata server and data servers are designed
   and implemented as a unit and interoperability needs to be assured
   between clients MUST and metadata-data server pairs, developed
   independently.  This is the model used for the files layout.

   In some cases, there may be able no control protocol other than the
   storage Another possibility, not so far realized, is for the
   definition of a control protocol to access be specified in a file directly through standards-track
   document.  There are two subcases to consider:

   o  A new layout type includes a definition of a particular control
      protocol whose use is obligatory for metadata serverss and storage
      devices implementing the layout type.  In this case the
      interoperability model is similar to the first case above and the
      defining document should assure interoperability among metadata
      servers, storage devices, and clients developed independently.

   o  A control protocol is defined in a standards-track document which
      meets the control protocol requirements for one of the existing
      layout types.  In this case, the new document's job is to assure
      interoperability between metadata servers and storage devices
      developed separately.  The existing definition document for the
      selected layout type retains the function of assuring
      interoperability between clients and a given collection of
      metadata servers and storage devices.  In this context,
      implementations that implement the new protocol are treated in the
      same way as those that use an internal control protocol or a
      functional equivalent.

3.1.  Protocol REQUIREMENTS

   The REQUIREMENTS of such interactions between the metadata server and not
   the storage device.  I.e., devices are:

   (1)  The metadata server MUST be able to service the client's I/O
        requests if the client decides to make such requests to the
        metadata server instead of to the storage device.  The metadata
        server must be able to retrieve the data from the constituent
        storage devices and present it back to the client.  A corollary
        to this is that even though the metadata server has successfully
        given the client
        via normal NFSv4.1 operations. a layout, the client MAY still send I/O
        requests to the metadata server.

        Whether the metadata server allows access over other protocols
        (e.g., NFSv3, Server Message Block (SMB), etc) is strictly an
        implementation choice. choice, just as it is in the case of any other
        (i.e., non-pNFS-supporting) NFSv4.1 server.

   (2)  The metadata server MUST be able to restrict access to a file on
        the storage devices when it revokes a layout.  The metadata
        server typically would revoke a layout whenever a client fails
        to respond to a recall or fails to renew its a client's lease in time. is expired due to
        non-renewal.  It might also revoke the layout as a means of
        enforcing a change in locking state or access permissions that
        the storage device cannot directly enforce with the
        client. enforce.

        Effective revocation may require client co-operation in using a
        particular stateid (files layout) or principal (e,g., flexible
        files layout) when performing I/O.

   (3)  Storage devices  A pNFS impelementation MUST NOT remove NFSv4.1's access
        controls: ACLs and file open modes.  While Section 12.9 of
        [RFC5661] specifically lays this burden on the combination of
        clients, storage devices, and the metadata server, depending on
        the implementation, there might be a requirement that the
        metadata server update the storage device such that it can
        enforce security.

        The file layout requires the storage device to enforce access
        whereas the flex file layout requires both the storage device
        and the client to enforce security.

   (4)  Locking MUST be respected.

   (5)  The metadata server and the storage devices MUST agree on
        attributes like modify time, the change attribute, and the end-
        of-file (EOF) position.

        Note that "agree" here means

        (a)  "Agree" in the sense that some while state changes need not
             be propagated immediately, although all changes SHOULD they must be propagated promptly.

   Note that there when
             accessed by the client.  This access is no typically in
             response to a GETATTR of those attributes.

        (b)  A particular storage device might be striped such it knows
             nothing about the EOF position.  It still meets the
             requirement of agreeing on how these are implemented.
   While that fact with the File Layout Type metadata

        (c)  Both clock skew and network delay can lead to the metadata
             server and the storage device having different concepts of
             the time attributes.  As long as those differences can be
             accounted for what is presented to the client in a GETATTR,
             then the two "agree".

        (d)  A LAYOUTCOMMIT requires that storage device generated
             changes in attributes need be reflected in the metadata
             server by the completion of the operation.

   These requirements may be satisfied in different ways by different
   layout types.  As an example, while the file layout type does use the
   stateid to fence off the client, there is no requirement that other Layout Types
   layout types use this stateid approach.  But the other Layout Types MUST

   Each new standards-track document for a layout types MUST address how
   the client, metadata server, and storage devices interact to meet
   these requirements.

3.2.  Non-protocol Requirements  Undocumented Protocol REQUIREMENTS

   In gathering the requirements from Section 12 of [RFC5661], there are
   some which are notable in their absence:

   (1)  Storage device  Clients MUST honor the byte range restrictions present in
        the layout.  I.e., if the layout only provides access NOT perform I/O to the
        first 2 MB of storage device if they do
        not have layouts for the file, then any access after that MUST NOT be
        granted. files in question.

   (2)  The enforcement of authentication and authorization so that
        restrictions that would  Clients MUST be enforced by allowed to perform I/O to the metadata server are
        also enforced by the storage device.  Examples include both
        export access checks and
        even if the they already have a LAYOUT.  A layout has an iomode type might
        discourage such I/O, but it can not forbid it.

   (3)  Clients MUST NOT perform I/O operations outside of
        LAYOUTIOMODE4_READ, then if the client attempts to write, specified
        ranges in the I/
        O may layout segment.

   (4)  Clients MUST NOT perform I/O operations which would be rejected.

        While storage devices should make such checks on
        inconsistent with the iomode specified in the layout
        iomode, [RFC5661] does not mandate that all Layout Types have to
        make such checks.

   (3) segments it

   (5)  The metadata server MUST be able to do allocation and
        deallocation of storage.  I.e., creating and deleting files.

   Of these,

   Under the first two file layout type, the storage devices are of concern able to this draft and Layout Types
   SHOULD honor them if at meet all possible,

3.3.  Editorial Requirements

   In addition to
   of these protocol requirements, there are two editorial
   requirements for drafts that present a new Layout Type.  At a
   minimum, requirements.  However, this is not the specification needs case with the other
   known layout types, Instead, the burden is shifted to address: both:

   (1)  The approach the new Layout Type takes towards fencing clients
        once client itself.

   (2)  The interaction of the metadata server determines that and the layout is revoked.

   (2) client.

   The security considerations of metadata server is responsible for giving the new Layout Type.

   While these could client enough
   information to make informed decisions and for trusting the client
   implementation to do so.  This communication would be envisioned as one through the
   callback operatios available to the metadata server, e.g., recalling
   a layout, a delegation, etc.

3.3.  Editorial Requirements

   This section discusses how the protocol requirements discussed above
   need to be addressed in documents specifying a new layout type.
   Depending on the interoperability model for the layout type in
   question, this may involve the imposition of layout-type-specific
   requirements that ensure appropriate interoperability of pNFS
   components which are developed separately.

   The specification of the fencing
   issue might be layout type needs to make clear how the only security issue,
   client, metadata server, and storage device act together to meet the
   protocol requirements discussed previously.  If the document does not
   impose implementation requirements sufficient to ensure that these
   semantic requirements are met, it is recommended not appropriate for the working
   group to deal
   with them separably.

   The specification of allow the document to move forward.

   Some examples include:

   o  If the metadata server does not have a means to invalidate a
      stateid issued to the storage device to keep a particular client
      from accessing a specific file, then the Layout Type should discuss layout type spefication
      has to document how the client, metadata server, and storage device act together server is going to meet fence the protocol
   requirements.  I.e., if
      client from access to the file on that storage device cannot enforce mandatory
   byte-range locks, then how can device.

   o  If the metadata server and implements mandatory byte-range locking
      when accessed directly by the client
   interact with client, it must do so when data is
      read or written using the layout to enforce those locks? designated storage protocol.

4.  Implementations in  Specifications of Existing Layout Types

   This section is not normative with regards to each of the presented
   types.  This document does not update the specification of either the
   block layout type (see [RFC5663]) or the object layout type (see
   [RFC5664]).  Nor does it update Section 13 of [RFC5661], but rather
   Section 12 of that document.  In other words, it is the pNFS
   requirements being updated, not the specification of the file layout

4.1.  File Layout Type

   Not surprisingly,

   Because the File Layout Type storage protocol is a subset of NFSv4.1, the semantics of
   the file layout type comes closest to the normal semantics of NFSv4.1. NFSv4.1 in the
   absence of pNFS.  In particular, the stateid and principal used for I/O I
   /O MUST have the same effect and be subject to the same validation on
   a data server as it would if the I/O was were being performed on the
   metadata server itself in the absence itself.  The same set of pNFS. validations apply whether
   pNFS is in effect or not.

   And while for most implementations the storage devices can do storage devices can do the
   following validations:

   (1)  client holds a valid layout,

   (2)  client I/O matches the layout iomode, and,

   (3)  client does not go out of the byte ranges,

   these are each presented as a "SHOULD" and not a "MUST".  Actually,
   the first point is presented as both:

   "MUST":  in Section 13.6 of [RFC5661]

      "As described in Section 12.5.1, a client MUST NOT send an I/O to
      a data server for which it does not hold a valid layout; the data
      server MUST reject such an I/O."

   "SHOULD":  in Section 13.8 of [RFC5661]

      "The iomode need not be checked by the data servers when clients
      perform I/O.  However, the data servers SHOULD still validate that
   following validations:

   o client holds a valid layout,

   o  client I/O matches the layout iomode, and,

   o and return an error if the client
      does not go out of the byte ranges,

   these are each presented as a "SHOULD" and not a "MUST". not."

   However, it is just these layout specific checks that are optional,
   not the normal file access semantics.  The storage devices MUST make
   all of the required access checks on each READ or WRITE I/O as
   determined by the NFSv4.1 protocol.  If the metadata server would
   deny a READ or WRITE operation on a file due to its ACL, mode
   attribute, open access mode, open deny mode, mandatory byte-range
   lock state, or any other attributes and state, the storage device
   MUST also deny the READ or WRITE operation.  And note that while the
   NFSv4.1 protocol does not mandate export access checks based on the
   client's IP address, if the metadata server implements such a policy,
   then that counts as such state as outlined above.

   As the

   The data filehandle provided by the PUTFH operation and to the
   stateid in data
   server is sufficient to ensure that for the subsequent READ or WRITE
   operation are used to ensure in the compound, that the client has a valid layout for the
   I/O being performed, performed.

   Finally, the client data server can check the stateid presented in the READ
   or WRITE operation to see if that stateid has been rejected by the
   metadata server such to cause the I/O to be fenced off for access fenced.  Whilst it might
   just be the open owner or lock owner on that client being fenced, the
   client should take the NFS4ERR_BAD_STATEID error code to a specific mean it has
   been fenced from the file via and contact the invalidation of
   either key. metadata server.

4.2.  Block Layout Type

   With the Block Layout Type, block layout type, the storage devices are not guaranteed to
   be able to enforce file-based security.  Typically, storage area
   network (SAN) disk arrays and SAN protocols provide access control
   mechanisms (e.g., Logical Unit Number (LUN) mapping and/or masking),
   which operate at the granularity of individual hosts, not individual
   blocks.  Access to block storage is logically at a lower layer of the
   I/O stack than NFSv4, and hence NFSv4 security is not directly
   applicable to protocols that access such storage directly.  As such,
   Section 2.1 [RFC5663] is very careful to define that in specifies that:

      "in environments where pNFS clients cannot be trusted to enforce
      such policies, pNFS Block Layout
   Types block layout types SHOULD NOT be used.

   The implication here is that used."

   As a result of these granularity issues, the security burden has been
   shifted from the storage devices to the client.  It is the responsibility  Those deploying
   implementations of the
   administrator doing the deployment this layout type need to trust be sure that the client
   implementation.  However, this
   implementation can be trusted This is not a new sort of requirement when it comes
   in the context of SAN protocols, protocols.  In such environments, the client is
   expected to provide block-based protection.

   This implication shift of the burden also extends to ACLs, locks, locks and layouts.  The
   storage devices might are not be able to enforce any of these and the burden
   is pushed to the client to make the appropriate checks before sending
   I/O to the storage devices.  As an  For example, if the metadata server uses may use a layout
   iomode for only allowing reading to enforce a mandatory read-
   only read-only lock, then
   In such cases, the client has to honor support that intent use by not sending
   WRITEs to the storage devices.  The basic fundamental issue here is that
   the storage device can be is treated by this layout type as a local dumb disk such that once
   disk.  Once the client has access to the storage device, it is able
   to perform either both READ or and WRITE I/O to the entire storage device.  The
   byte ranges in the layout, any locks, the layout iomode, etc, can
   only be enforced by the client.

   While  Therefore, the Block Layout Type does support client is required to
   provide that enforcement.

   In the context of fencing off of the client upon revoking revocation of a layout, the above restrictions
   layout, these limitations come into play again: again, i.e., the granularity
   of the fencing can only be at the host/logical-unit level.  Thus, if
   one of a client's layouts is unilaterally revoked by the server, it will
   effectively render useless *all* revoke all of the client's layouts for files located on
   the storage units comprising the logical volume.  This may render useless extend to
   the client's layouts for files in other file systems.  Clients need
   to be prepared for such revocations and reacquire layouts as needed.

4.3.  Object Layout Type

   The Object Layout Type focuses

   With the object layout type, security checks to occur during the
   allocation of the layout.  The client will typically ask for a layout
   for each byte-range layouts
   covering all of the file and may do so for either READ or READ/WRITE.
   This enables it to do subsequent I/O operations without the need to
   obtain layouts for specific byte ranges.  At that time, the metadata
   server should verify permissions against the layout iomode, the outstanding locks, the file
   mode bits or ACLs, etc.  As the client may be acting for multiple
   local users, it MUST authenticate and authorize the user by issuing
   respective OPEN and ACCESS calls to the metadata server, similar to
   having NFSv4 data delegations.

   Upon successful authorization, inside the layout, the client receives within the layout
   a set of object capabilities allowing it I/O access to the specified
   objects corresponding to the requested iomode.  These capabilities
   are used to enforce access control and locking semantics at the
   storage devices.  Whenever one of the following occur on the metadata server detects one of:

   o  the permissions on the object change,

   o  a conflicting mandatory byte-range lock is granted, or

   o  a layout is revoked and reassigned to another client,

   then it the metadate server MUST change the capability version attribute
   on all objects comprising the file to implicitly in order to invalidate any
   outstanding capabilities before committing to one of these changes.

   When the metadata server wishes to fence off a client to a particular
   object, then it can use the above approach to invalidate the
   capability attribute on the given object.  The client can be informed
   via the storage device that the capability has been rejected and is
   allowed to fetch a refreshed set of capabilities, i.e., re-acquire
   the layout.

5.  Summary

   In the three published Layout Types, layout types, the burden of enforcing the
   security of NFSv4.1 can fall to either the storage devices (Files), (files),
   the client (Blocks), (blocks), or the metadata server (Objects). (objects).  Such
   decisions seem to be forced choices
   are conditioned by the native capabilities of the storage devices -
   if a real control protocol can be implemented, then the burden can be
   shifted primarily to the storage devices.

   But as

   In the context of this document, we have seen, treat the control protocol is actually as a
   set of requirements.  And as new Layout Types layout types are published, the enclosing
   defining documents minimally MUST address:

   (1)  The fencing of clients after a layout is revoked.

   (2)  The security implications of the native capabilities of the
        storage devices with respect to the requirements of the NFSv4.1
        security model.

   In addition, these defining documents need to make clear how other
   semantic requirements of NFSv4.1 (e.g., locking) are met in the
   context of the proposed layout type.

6.  Security Considerations

   This section does not deal directly with security considerations for
   existing or new layout types.  Instead, it provides a general
   framework for understating security-related issues within the pNFS
   framework.  Specific security considerations will be addressed in the
   Security Considerations sections of documents specifying layout

   The layout type specification must ensure that only data accesses
   consistent with the NFSV4.1 security model are allowed.  It may do
   this directly, by providing that appropriate checks be performed at
   the time the access is performed.  It may do it indirectly by
   allowing the client or the storage device to be responsible for
   making the appropriate checks.  In the latter case, I/O access writes
   are reflected in layouts and the layout type must provide a way to
   prevent inappropriate access due to permissions changes between the
   time a layout is granted and the time the access is performed.

   The metadata server MUST be able to fence off a client's access to a
   the data file stored on a storage device.  When it revokes the layout, the
   client's access MUST be terminated at the storage devices.  The
   client the has the opportunity to re-acquire the layout and perform
   the security check in the context of the newly current access

7.  IANA Considerations

   This document has no actions for IANA.

8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", March 1997.

   [RFC5661]  Shepler, S., Eisler, M., and D. Noveck, "Network File
              System (NFS) Version 4 Minor Version 1 Protocol", RFC
              5661, January 2010.

   [RFC5663]  Black, D., Fridella, S., and J. Glasgow, "pNFS Block/
              Volume Layout", RFC 5663, January 2010.

   [RFC5664]  Halevy, B., Welch, B., and J. Zelenka, "Object-Based
              Parallel NFS (pNFS) Operations", RFC 5664, January 2010.

8.2.  Informative References

              Halevy, B. and T. Haynes, "Parallel NFS (pNFS) Flexible
              File Layout", draft-ietf-nfsv4-flex-files-02 draft-ietf-nfsv4-flex-files-11 (Work In
              Progress), October 2014. July 2017.

   [Lustre]   Faibish, S. and P. Tao, "Parallel NFS (pNFS) Lustre Layout
              Operations", draft-faibish-nfsv4-pnfs-lustre-layout-07
              (Work In Progress), April 2014.

Appendix A.  Acknowledgments

   Dave Noveck provided an early review that sharpened the clarity of
   the definitions.  He also provided a more comprehensive review of the

Appendix B.  RFC Editor Notes

   [RFC Editor: please remove this section prior to publishing this
   document as an RFC]

   [RFC Editor: prior to publishing this document as an RFC, please
   replace all occurrences of RFCTBD10 with RFCxxxx where xxxx is the
   RFC number of this document]

Author's Address
   Thomas Haynes
   Primary Data, Inc.
   4300 El Camino Real Ste 100
   Los Altos, CA  94022

   Phone: +1 408 215 1519