Subject: [nfsv4] I-D Action: draft-ietf-nfsv4-flex-files-05.txt

From: [email-masked]
Date: Mon, 09 Feb 2015 17:39:47 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Network File System Version 4 Working Group of the IETF.

        Title           : Parallel NFS (pNFS) Flexible File Layout
        Authors         : Benny Halevy
                          Thomas Haynes
	Filename        : draft-ietf-nfsv4-flex-files-05.txt
	Pages           : 32
	Date            : 2015-02-09

Abstract:
   The Parallel Network File System (pNFS) allows a separation between
   the metadata (onto a metadata server) and data (onto a storage
   device) for a file.  The Flexible File Layout Type is defined in this
   document as an extension to pNFS to allow the use of storage devices
   in a fashion such that they require only a quite limited degree of
   interaction with the metadata server, using already existing
   protocols.  Client side mirroring is also added to provide
   replication of files.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-nfsv4-flex-files-05

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-flex-files-05

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Tom Haynes <[email-masked]>
Date: Mon, 9 Feb 2015 17:45:28 -0800

Hi,

I wanted to push a new version of the draft to reflect the changes made due
to the various reviews.

I’ll need to send out another version to take care of the following open issues:

1) Can loosely coupled be kerberized?
2) Dave’s request to describe locking.
3) Olga’s point on the use of the Anonymous Stateid for loosely coupled access.

But I did want to see if the text describing synthetic uids and gids helped explain
the various discussions.

Thanks,
Tom

> On Feb 9, 2015, at 5:39 PM, [email-masked] wrote:
> 
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 10 Feb 2015 01:56:58 +0000

Thanks, Tom.  Given the set of changes during last call, once you have all the changes in we will run another last call to close completely.

Spencer

> -----Original Message-----
> From: nfsv4 [mailto:[email-masked]] On Behalf Of Tom Haynes
> Sent: Monday, February 9, 2015 5:45 PM
> To: NFSv4
> Subject: Re: [nfsv4] I-D Action: draft-ietf-nfsv4-flex-files-05.txt
> 
> Hi,
>

---

From: Tom Haynes <[email-masked]>
Date: Tue, 10 Feb 2015 06:59:15 -0800

Agreed, I was expecting another round. :-)

> On Feb 9, 2015, at 5:56 PM, Spencer Shepler <[email-masked]> wrote:
> 
>

---

From: Olga Kornievskaia <[email-masked]>
Date: Fri, 13 Feb 2015 11:34:58 -0800

Hi Tom,

Thanks for the new draft. A few questions/comments:

1. Can the definition of "fencing" be more accurate given that in loosely
couple model we can't fence a "specific" client? So change it to: "fencing:
is when the metadata server prevents the storage devices from processing IO
to a specific file".

2. It is still confusing that "loosely coupling" is defined specifically as
something that doesn't have a control protocol. Yet, in section 2, it says
"with a loose coupling, the only control protocol might be a version of
NFS". Perhaps that statement can be made to be a part of the definition
itself. In my understanding, in loosely coupled model, an MDS needs to
communicate with a DS to get the file handles, why is that not a "control
protocol"?

3. I still don't see the necessity of introducing synthetic uids/gids. If
we were to keep normal usage (i.e., file owned by the user and give 644
permissions), then to fence the owner, just as in the draft proposal, a uid
can be changed to say 0, or a random uid, and permissions can be changed to
000 to fence of others. What is it that can be achieved with synthetic
uids/gids that can't be done with that I just described?

4. In section 4.1, "If ffdv_version equals 4 then the server MUST set
ffdv_minorversionto one of the NFSv4 minor version numbers and the client
MUST access the storage device using NFSv4". Would be it more accurate to
say "... and the client MUST access the storage device using the specified
protocol"? Just saying NFSv4 is confusing. Or is the intention to allow for
something like if ffdv_version=4 and ffdv_minorversion=2 but the client is
allowed to choose 4.1 and not 4.2 protocol?

5. In section 5.1, "the fl_mirrors field is the array of mirrored storage
devices which provide the storage for the current stripe, see Figure 1". Is
it "current stripe" or "layout segment"? Does Figure 1 portray an example
of layout with 1 segment?

6. In section 15, "the metadata enforces the file access-control policy at
LAYOUTGET time". Is this in addition to the normal access control that's
done at OPEN?

Thanks.

---

From: Tom Haynes <[email-masked]>
Date: Fri, 13 Feb 2015 13:41:03 -0800

> On Feb 13, 2015, at 11:34 AM, Olga Kornievskaia <[email-masked]> wrote:
> 
> Hi Tom,
> 
> 1. Can the definition of "fencing" be more accurate given that in loosely couple model we can't fence a "specific" client? So change it to: "fencing: is when the metadata server prevents the storage devices from processing IO to a specific file”.

I’d prefer to keep a consistent definition of fencing across the documents and spell out the divergence here.

> 
> 2. It is still confusing that "loosely coupling" is defined specifically as something that doesn't have a control protocol. Yet, in section 2, it says "with a loose coupling, the only control protocol might be a version of NFS". Perhaps that statement can be made to be a part of the definition itself. In my understanding, in loosely coupled model, an MDS needs to communicate with a DS to get the file handles, why is that not a "control protocol"?
> 

Looks like I did not apply all of Dave’s suggestions.

   loose coupling:  is when the metadata server and the storage devices
      do not have a control protocol present.

becomes

   loose coupling:  is when the metadata server and the storage devices
      do not have a purpose build control protocol present. As such the
      only interaction been them is via an established protocol.

And

2.  Coupling of Storage Devices

   The coupling of the metadata server with the storage devices can be
   either tight or loose.  In a tight coupling, there is a control
   protocol present to manage security, LAYOUTCOMMITs, etc.  With a
   loose coupling, the only control protocol might be a version of NFS.
   As such, semantics for managing security, state, and locking models
   MUST be defined.

becomes

2.  Coupling of Storage Devices

   The coupling of the metadata server with the storage devices can be
   either tight or loose.  In a tight coupling, there is a control
   protocol , a purpose-built protocol (not specified here),
   present to manage security, LAYOUTCOMMITs, etc.  With a
   loose coupling, the only control protocol might be a version of NFS.
   As such, semantics for managing security, state, and locking models
   MUST be defined.

> 3. I still don't see the necessity of introducing synthetic uids/gids. If we were to keep normal usage (i.e., file owned by the user and give 644 permissions), then to fence the owner, just as in the draft proposal, a uid can be changed to say 0, or a random uid, and permissions can be changed to 000 to fence of others. What is it that can be achieved with synthetic uids/gids that can't be done with that I just described?

Perhaps the term should be changed from synthetic uid to synthetic owner?

I.e., if we use kerberos, we need a principal here and the non-stringified owner supplies that.

> 
> 4. In section 4.1, "If ffdv_version equals 4 then the server MUST set ffdv_minorversionto one of the NFSv4 minor version numbers and the client MUST access the storage device using NFSv4". Would be it more accurate to say "... and the client MUST access the storage device using the specified protocol"? Just saying NFSv4 is confusing. Or is the intention to allow for something like if ffdv_version=4 and ffdv_minorversion=2 but the client is allowed to choose 4.1 and not 4.2 protocol?

If
   ffdv_version equals 4 then the server MUST set ffdv_minorversion to
   one of the NFSv4 minor version numbers and the client MUST access the
   storage device using NFSv4.

becomes

If
   ffdv_version equals 4 then the server MUST set ffdv_minorversion to
   one of the NFSv4 minor version numbers and the client MUST access the
   storage device using NFSv4 and that minor version. In order to specify
   different NFSv4 minor versions, there each allowed variant MUST
   have an entry in the ffda_versions array.

> 
> 5. In section 5.1, "the fl_mirrors field is the array of mirrored storage devices which provide the storage for the current stripe, see Figure 1". Is it "current stripe" or "layout segment”?

“current layout segment"

> Does Figure 1 portray an example of layout with 1 segment?

Yes

> 
> 6. In section 15, "the metadata enforces the file access-control policy at LAYOUTGET time". Is this in addition to the normal access control that's done at OPEN? 

Thanks for pointing this remnant of the old object layout capabilities. This part should come out.

> 
> Thanks.
> 
> On Mon, Feb 9, 2015 at 8:39 PM, <[email-masked] <mailto:[email-masked]>> wrote:
> 
> https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/ <https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/>
> 
> http://tools.ietf.org/html/draft-ietf-nfsv4-flex-files-05 <http://tools.ietf.org/html/draft-ietf-nfsv4-flex-files-05>
> 
> http://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-flex-files-05 <http://www.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-flex-files-05>
> 
> 
> until the htmlized version and diff are available at tools.ietf.org <http://tools.ietf.org/>.
> 
> ftp://ftp.ietf.org/internet-drafts/ <ftp://ftp.ietf.org/internet-drafts/>
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/nfsv4 <https://www.ietf.org/mailman/listinfo/nfsv4>
>

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 13 Feb 2015 17:35:25 -0500

Hi Olga,

On Fri, Feb 13, 2015 at 2:34 PM, Olga Kornievskaia <[email-masked]> wrote:
> 3. I still don't see the necessity of introducing synthetic uids/gids. If we
> were to keep normal usage (i.e., file owned by the user and give 644

What then, if a process on a different client calls LAYOUTGET and you
restore the uid to the original owner (or the gid to the original
group owner)? Suddenly, the old layout is "restored" for all clients
whether or not they have requested another layout for that file.

The value of the synthetic uids and gids lies in the fact that they
are temporary and can be made more or less non-recurring in time, so
that fenced clients are forced to request a new layout before they can
perform I/O.

Cheers
  Trond