Subject: [nfsv4] NFSv4.2 vs pNFS

From: Christoph Hellwig <[email-masked]>
Date: Tue, 19 Nov 2013 18:17:34 +0100

While reading through the current NFSv4.2 drafts I have few questions
vs the interaction with pnfs.

For one rfc3530.txt generally seems to define the file layout as a
"NFSv4.1" file layout, allowing access using a subset of the 4.1 protocol.

There is no language in the 4.2 defintining any sort of NFS v4.2 file
layout, just a few implication that READ_PLUS should be fine on 4.2,
without explicitly allowing it nor definining a protocol for DS access.

There also is no language saying if SEEK or WRITE_PLUS should be allowed
against a DS, and how a DS should respond if it can't support the operation.
E.g. a hole punch often is a metadata centric operation that I would not
accept to be handled by a DS unless the cluster has a fully symmetric
architecture.

I'd also love to know what the expected semantics for block or object
layout to a file undergoing offloaded operations are.

---

From: "Noveck, David" <[email-masked]>
Date: Tue, 19 Nov 2013 14:45:30 -0500

I agree that there is a lot missing on this topic in the current specs.

Looking at the current spec, it seems that people did have some thoughts about how this might be addressed but the spec text reflecting that was never finished.  As an example, look at the errors beginning "NFS4ERR_PNFS_".  They are listed as being returned by COPY-related ops and {READ,WRITE}_PLUS but there is no definition of what these errors mean.  There is no text specifying when particular  ops might return these, although there is some text specifying some situations in which READ_PLUS *MUST NOT* return NFS4ERR_PNFS_IO_HOLE.

It's unfortunate that we didn't address this sort of issue before the decision to include these features in NFSv4.2 was made.  

-----Original Message-----
From: nfsv4 [mailto:[email-masked]] On Behalf Of Christoph Hellwig
Sent: Tuesday, November 19, 2013 12:18 PM
To: [email-masked]
Subject: [nfsv4] NFSv4.2 vs pNFS

While reading through the current NFSv4.2 drafts I have few questions vs the interaction with pnfs.

For one rfc3530.txt generally seems to define the file layout as a "NFSv4.1" file layout, allowing access using a subset of the 4.1 protocol.

There is no language in the 4.2 defintining any sort of NFS v4.2 file layout, just a few implication that READ_PLUS should be fine on 4.2, without explicitly allowing it nor definining a protocol for DS access.

There also is no language saying if SEEK or WRITE_PLUS should be allowed against a DS, and how a DS should respond if it can't support the operation.
E.g. a hole punch often is a metadata centric operation that I would not accept to be handled by a DS unless the cluster has a fully symmetric architecture.

I'd also love to know what the expected semantics for block or object layout to a file undergoing offloaded operations are.
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 29 Nov 2013 17:51:49 +0000

> It's unfortunate that we didn't address this sort of issue before the decision to include these features in NFSv4.2 was made.  

We did, we just didn't record it.

> There is no language in the 4.2 defintining any sort of NFS v4.2 file layout, just a few implication that READ_PLUS should be fine on 4.2, without explicitly allowing it nor definining a protocol f…

There is no new NFSv4.2 file layout.

> E.g. a hole punch often is a metadata centric operation that I would not accept to be handled by a DS unless the cluster has a fully symmetric architecture.

We did discuss this in the weekly meetings and I believe we removed text for this.

Yes, we did not extend the following:

14.10.4.1.  Additional pNFS Implementation Information

   With pNFS, the semantics of using READ_PLUS remains the same.  Any
   data server MAY return a hole or ADH result for a READ_PLUS request
   that it receives.  When a data server chooses to return such a
   result, it has the option of returning information for the data
   stored on that data server (as defined by the data layout), but it
   MUST NOT return results for a byte range that includes data managed
   by another data server.

to either SEEK or WRITE_PLUS, but the intent is the same.

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 30 Nov 2013 17:31:50 +0100

> to either SEEK or WRITE_PLUS, but the intent is the same.

How does a data server know if it should follow 4.1 or 4.2 semantics?

Also RFC 5661 is very specific that the protocol to speak to the data
server is exactly NFSv4.1, and there is no language lifting that limit
in the 4.2 draft spec.

To me the way this is introduced seems rather inconsistent.