Subject: [nfsv4] FW: I-D ACTION:draft-faibish-nfsv4-pnfs-access-permissions-check-03.txt

From: <[email-masked]>
Date: Mon, 12 Jul 2010 12:49:08 -0400

This is the revised permissions check draft - it actually deals with any
circumstance under which a client cannot access a pNFS data server and
wants to report that inaccessibility.

Thanks,
--David

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Monday, July 12, 2010 12:45 PM
To: [email-masked]
Subject: I-D
ACTION:draft-faibish-nfsv4-pnfs-access-permissions-check-03.txt 

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title		: pNFS Access Permissions Check
	Author(s)	: S. Faibish, D. Black, M. Eisler, J. Glasgow
	Filename	:
draft-faibish-nfsv4-pnfs-access-permissions-check-03.txt
	Pages		: 10
	Date		: 2010-7-12
	
This document extends the pNFS protocol to communicate errors caused 
   by inability to access data servers referenced by layouts, including 
   checks performed by both clients and the MDS. The extension provides 
   means for clients to communicate client-detected access denial errors

   to the MDS, including the case in which a client requests direct NFS 
   access via the MDS that the MDS cannot perform.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-faibish-nfsv4-pnfs-access-perm
issions-check-03.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: Tom Haynes <[email-masked]>
Date: Mon, 12 Jul 2010 15:19:26 -0500

> wants to report that inaccessibility.

1)  This:

   To the extent that an 
   MDS can determine whether storage devices are accessible to clients, 
   an MDS SHOULD NOT include a storage device in any pNFS layouts sent 
   to a client that cannot access that storage device. At a minimum, the 
   server SHOULD perform these storage device accessibility checks 
   before exporting a filesystem that supports pNFS and when the device 
   configuration for such an exported filesystem is changed (e.g., to 
   add a storage device).

implies to me that the MDS has to keep track of 
LAYOUT4_RET_REC_FSID_NO_ACCESS
and LAYOUT4_RET_REC_FILE_NO_ACCESS layout return types per client. I.e., 
once it
knows a client has problems with a specific storage device, it should 
avoid using that
device again.

Given that we need this mechanism for the client to report errors, how 
then does the server
know when it can start using these storage devices again? Even if the 
MDS knows a positive
change took place, it has to rely on the client to do the checking.

Is this a difference between MUST and SHOULD? I.e., does having SHOULD 
mean that
the MDS can hand out the storage devices again to see if the client can 
suddenly start
using them again?

2) Is it NFS4ERR_PERM or NFS4ERR_ACCESS for access permission denial?

   o  NFS4ERR_PERM SHOULD be used for access permission denial; and 

 From 5661:

15.1.6.1. NFS4ERR_ACCESS (Error Code 13)

   Indicates permission denied.  The caller does not have the correct
   permission to perform the requested operation.  Contrast this with
   NFS4ERR_PERM (Section 15.1.6.2), which restricts itself to owner or
   privileged-user permission failures, and NFS4ERR_WRONG_CRED
   (Section 15.1.6.4), which deals with appropriate permission to delete
   or modify transient objects based on the credentials of the user that
   created them.

15.1.6.2. NFS4ERR_PERM (Error Code 1)

   Indicates requester is not the owner.  The operation was not allowed
   because the caller is neither a privileged user (root) nor the owner
   of the target of the operation.

Since this document talks about mount issues, I went back to RFC 1813
and the MOUNT protocol. MNT can return MNT3ERR_ACCES if
the client does not have access rights to the export.

I think NFS4ERR_ACCESS is more consistent with prior protocols
than NFS4ERR_PERM.

Going back to this draft, in section 3.3, I see:

   There are two NO_ACCESS layoutreturn_type4 values that indicate lack 
   of storage device access, LAYOUT4_RET_REC_FSID_NO_ACCESS and 
   LAYOUT4_RET_REC_FILE_NO_ACCESS. 

and 

   An NFS error (nfsstat4) is 
   included in the layoutreturn data structures for these two types to 
   distinguish access permission problems from device inaccessibility: 

I think access has been overloaded here and to clarify things, 
NFS4ERR_PERM is selected
over NFS4ERR_ACCESS.

The only other reasons I can see for using NFS4ERR_PERM instead of 
NFS4ERR_ACCESS
are related to security:

a) if the user credentials were insufficient, i.e., kerberized access to 
the storage device failed.

b) Section 13.12 of 5661:

   If the metadata server would
   deny a READ or WRITE operation on a file due to its ACL, mode
   attribute, open access mode, open deny mode, mandatory byte-range
   lock state, or any other attributes and state, the data server MUST
   also deny the READ or WRITE operation.

Which seems to point out a need for error codes for:

a) No access granted (mount for files, block devices have other means)
b) Permission denied for the operation.
c) Permission denied because of security.

The difference between b) and c) is that b) is per fileid and c) is per 
fsid. So a MDS could
still use the storage device in the layout for b), but should avoid 
using it for c).

3) I find:

   An NFS error (nfsstat4) is 
   included in the layoutreturn data structures for these two types to 
   distinguish access permission problems from device inaccessibility: 

   o  NFS4ERR_PERM SHOULD be used for access permission denial; and 

   o  NFS4ERR_NXIO SHOULD be used for inability to access a device. 

   Other NFS errors MAY be used when they are appropriate. All uses of 
   these two layout return types that report errors SHOULD be logged by 
   the client. 

to be under-specified. What are the other errors that a server can see 
and how is it
supposed to react to those errors?

I'd like to see language about which errors are MANDATORY to be 
supported and which
are OPTIONAL. I know I can read the above to see that there are only two 
MANDATORY
ones, but I can also read it to see that all are MANDATORY.

I don't want clients shoehorning every error code back into these two. 
And I do want clarification
on what a server should do with the OPTIONAL codes. I.e. is it free to 
reuse those storage
devices the next time that client asks for a layout?

4)

What if the storage device A returns NFS4ERR_STALE to the client while 
storage device
B returns NFS4_OK for an operation on the same layout? But for a 
different file with the
same layout, it is given NFS4_OK from both DSs. This isn't necessarily 
either a permission issue nor
a device inaccessibility issue. (It could be either: the export was 
changed or the filesystem
indicated by the filehandle does not exist. It could also just mean that 
the indicated file does
not exist.)

Would this be where LAYOUT4_RET_REC_FILE_NO_ACCESS and NFS4ERR_NXIO is 
appropriate?

Which leads to an even more interesting question, what constitutes a 
NFS4ERR_NXIO error?
As NFS4ERR_NXIO is defined by 5661 to not be a valid return code from 
any operation
or CB, I would take it to mean simply that the storage device is not 
responding to the client.
If the client got any error back from the storage device, then it could 
not use NFS4ERR_NXIO.

As the NFS4ERR_STALE could simply mean that the filehandle refers to a 
file which doesn't
exist, is it appropriate to inform the server to no longer use that 
storage device in the layouts
assigned to that client?

Where this is going is perhaps now is a good time to add more 
informative error codes from
the storage device to the client. This would in turn allow the client to 
send these back to the
MDS. I.e., there is a world of difference between this device (fsid) 
does not exist and this file
does not exist.

---

From: <[email-masked]>
Date: Tue, 13 Jul 2010 13:17:55 -0400

Tom,

Thanks for taking a look at this draft in such a prompt fashion.  All
four of your concerns turned up areas that needs attention.  Here are
the short answers (I've paraphrased your concerns as questions):

1) Is an MDS required to remember which clients have declared what
devices to be inaccessible?

That was not the intention; "To the extent that an MDS can determine
..." was supposed to be the text that would allow an implementation to
make its own decisions about when/whether/how to keep track of client
inability to access storage devices.  It looks like some stronger
language is needed to make that clear. FWIW, "SHOULD NOT" was
deliberately used instead of "MUST NOT", but that wasn't the entire
story.

then does the server
MDS knows a positive
> change took place, it has to rely on the client to do the checking.

The previous version of the draft added a callback for this purpose, but
that callback has been removed as not realistic at this juncture.  Hence
the practical answers to the above restart question probably involve
administrative intervention.

2) Should NFS4ERR_PERM or NFS4ERR_ACCESS be used for access permission
denial?

It's NFS4ERR_ACCESS, mea culpa; this will be fixed in -04.  In addition,
the list of I/O retry errors (when a failed storage device I/O is
retried via the MDS) that require clients to return the layout is
probably also wrong well beyond NFS4ERR_PERM vs. NFS4ERR_ACCESS :-(.

3) What are the other errors [beyond PERM/ACCESS or NXIO] that a server
can see for a client-inaccessible-storage-device and how is the server
supposed to react to those errors?

The current draft allows any error, although some of them don't make
much sense.  NFS4ERR_IO is an obvious error that is needed, as I expect
most SCSI errors to be mapped to that error.  As for which error code is
used, the server is supposed to log the error, but I don't think there's
any server processing that depends on which error the client sent (did
you see something of concern?).  OTOH, as part of digging into which I/O
retry errors require layout return, the list of errors allowed for an
inaccessible data server could be generated and added to the next
version of the draft.

4) What do the error codes actually mean? 

We need NFS4ERR_NXIO for inaccessible devices, so that error would need
to be unobsoleted for this purpose. Beyond that, see the response to 3)
- it looks like an explicit list is needed of errors that the client is
allowed to use when returning a layout or layouts because a storage
device in inaccessible.

As for a client getting NFS4ERR_STALE from a data server/storage device,
that error could be sent to the MDS or mapped to NFS4ERR_NXIO. IMHO,
it's a much better idea to just pass NFS4ERR_STALE on to the MDS so that
the MDS knows *exactly* what went wrong.  OTOH, NFS4ERR_NXIO would be
appropriate for inability to contact the data server (e.g., no TCP
connection, or RPC service is non-responsive).

OTOH, I'd prefer not to take the lid off the Pandora's box of passing
through SCSI errors, as the info involved doesn't always propagate up
the SCSI stack.

I'm glad this is a "Draft" ;-).

Thanks,
--David

> From: Tom Haynes [mailto:[email-masked]]
> Sent: Monday, July 12, 2010 4:19 PM
> To: Black, David
> Cc: [email-masked]
> Subject: Re: [nfsv4] FW: I-D

---

From: sfaibish <[email-masked]>
Date: Mon, 26 Jul 2010 12:55:10 -0400

Retry using gziped file as it was rejected the first time due to size  
limits.

/Sorin

On Mon, 26 Jul 2010 12:24:29 -0400, sfaibish <[email-masked]> wrote:

> Attached please find the presentation on Permission Access draft
> for the NFSv4 WG meeting of Wednesday at IETF 78.
>
> /Sorin
>
>
> On Mon, 12 Jul 2010 12:49:08 -0400, <[email-masked]> wrote:
>

-- 
Best Regards

Sorin Faibish
Corporate Distinguished Engineer
Network Storage Group

         EMC²
where information lives

Phone: 508-435-1000 x 48545
Cellphone: 617-510-0422
Email : [email-masked]