Subject: [nfsv4] Fwd: New Version Notification for draft-dnoveck-nfsv4-security-04.txt

From: David Noveck <[email-masked]>
Date: Fri, 24 Dec 2021 08:49:08 -0500

I've just posted security-04.   Thanks to Rick Macklem and Chuck Lever who
made important suggestions that I hope are correctly addressed in this
version.  An rfcdiff with -03 is not small but it is helpful to see what
has changed.

As previously discussed, I am proposing that the working group adopt this
draft as a working group document.   I expect Brian and Zahed to set the
timeline for that discussion.

Please let me know about your suggestions for -05.

---------- Forwarded message ---------
From: <[email-masked]>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]>

A new version of I-D, draft-dnoveck-nfsv4-security-04.txt
has been successfully submitted by David Noveck and posted to the
IETF repository.

Name:           draft-dnoveck-nfsv4-security
Revision:       04
Title:          Security for the NFSv4 Protocols
Document date:  2021-12-24
Group:          Individual Submission
Pages:          129
URL:
https://www.ietf.org/archive/id/draft-dnoveck-nfsv4-security-04.txt
Status:
https://datatracker.ietf.org/doc/draft-dnoveck-nfsv4-security/
Html:
https://www.ietf.org/archive/id/draft-dnoveck-nfsv4-security-04.html
Htmlized:
https://datatracker.ietf.org/doc/html/draft-dnoveck-nfsv4-security
Diff:
https://www.ietf.org/rfcdiff?url2=draft-dnoveck-nfsv4-security-04

Abstract:
   This document describes the core security features of the NFSv4
   family of protocols, applying to all minor versions.  The discussion
   includes the use of security features provided by RPC on a per-
   connection basis.

   This preliminary version of the document, is intended, in large part,
   to result in working group discussion regarding existing NFSv4
   security issues and to provide a framework for addressing these
   issues and obtaining working group consensus regarding necessary
   changes.

   When a successor document is eventually published as an RFC, it will
   supersede the description of security appearing in existing minor
   version specification documents such as RFC 7530 and RFC 8881.

The IETF Secretariat

---

From: Rick Macklem <[email-masked]>
Date: Fri, 24 Dec 2021 23:32:23 +0000

I think I will address each issue I see in this document separately, to try
and keep the discussion focused.
In Sec. 8.1 (page 64), it says:

 *  RFC8881 [8] contains the following, which is now superseded:

         Many servers implement owner-override semantics in which the
         owner of the object is allowed to override accesses that are
         denied by the ACL.  This may be helpful, for example, to allow
         users continued access to open files on which the permissions
         have changed.

      Regardless of the truth of the first sentence above, either when
      it was written or today, it needs to be stressed that the fact
      that a server manifests a particular behavior does not imply that
      it is valid according to the protocol specification.  In this
      case, the supposed "owner-override semantics" clearly are not
      valid, since they contradict the specification of both the mode-
      based and ACL-based approaches to file access authorization.

Well, I cannot see how the FreeBSD client (or other NFSv4 clients on
POSIX-like systems) can function without the "owner-override".
In POSIX, permission checking is done only when the POSIX open(2)
syscall is performed. If successful, I/O is expected to work, despite any
changes done to owner/owner_group/mode/ACL after the POSIX open(2)
while I/O is done on the open_fd.
--> Although NFS cannot be a POSIX compliant file system, allowing the
      owner of the file to do the Read/Write avoids frequent unexpected
      I/O failures due for owner/owner_group/mode/ACL changes.
--> I believe that "owner-override" has been common practice since NFSv2
      and should remain specified, even if it does appear to be in violation
      of other parts of the RFCs.
A new read-only per-file system attribute could be defined that tells a
client if "owner-override" is implemented, although not doing it will
probably cause a lot of grief for extant NFSv4 clients on POSIX like systems.

Does the Linux client expect "owner-override"?
If no one familiar with the Linux client answers this, I'll run a test on a Linux
mount.

rick

________________________________________
From: nfsv4 <[email-masked]> on behalf of David Noveck <[email-masked]>
Sent: Friday, December 24, 2021 8:49 AM
To: NFSv4; nfsv4-chairs; [email-masked]
Subject: [nfsv4] Fwd: New Version Notification for draft-dnoveck-nfsv4-security-04.txt

CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to [email-masked]

I've just posted security-04.   Thanks to Rick Macklem and Chuck Lever who made important suggestions that I hope are correctly addressed in this version.  An rfcdiff with -03 is not small but it is helpful to see what has changed.

As previously discussed, I am proposing that the working group adopt this draft as a working group document.   I expect Brian and Zahed to set the timeline for that discussion.

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]>>

Name:           draft-dnoveck-nfsv4-security
Revision:       04
Title:          Security for the NFSv4 Protocols
Document date:  2021-12-24
Group:          Individual Submission
Pages:          129
URL:            https://www.ietf.org/archive/id/draft-dnoveck-nfsv4-security-04.txt
Status:         https://datatracker.ietf.org/doc/draft-dnoveck-nfsv4-security/
Html:           https://www.ietf.org/archive/id/draft-dnoveck-nfsv4-security-04.html
Htmlized:       https://datatracker.ietf.org/doc/html/draft-dnoveck-nfsv4-security
Diff:           https://www.ietf.org/rfcdiff?url2=draft-dnoveck-nfsv4-security-04

Abstract:

The IETF Secretariat

---

From: Rick Macklem <[email-masked]>
Date: Sat, 25 Dec 2021 01:49:29 +0000

Rick Macklem wrote:
[stuff snipped]
> If no one familiar with the Linux client answers this, I'll run a test on a Linux
I ran the following little test program. It ran fine on local file systems and on
an NFSv4 mount for both the FreeBSD and Linux-5.15.1 clients, if
"owner-override" was applied to Read/Write.

If "owner-override" is turned off on the server, it fails for both the FreeBSD
and Linux-5.15.1 client NFSv4 mounts. (The write(2) fails for FreeBSD and the
close(2) fails for Linux, but that's ok. Unless there is an fsync(2), flushing writes
to the server can be delayed until close(2). Both had empty files after the
failure.)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/param.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <err.h>
#include <unistd.h>

char buf[1024 * 1024];

int
main(int argc, char *argv[])
{
	int fd, i;

	fd = open(argv[1], O_CREAT | O_RDWR, 0666);
	if (fd < 0)
		err(1, "cannot open %s", argv[1]);
	if (fchmod(fd, 0) < 0)
		err(1, "cannot chmod %s", argv[1]);
	for (i = 0; i < 10; i++)
		if (write(fd, buf, sizeof(buf)) != sizeof(buf))
			err(1, "cannot write");
	if (lseek(fd, SEEK_SET, 0) < 0)
		err(1, "cannot seek");
	for (i = 0; i < 10; i++)
		if (read(fd, buf, sizeof(buf)) != sizeof(buf))
			err(1, "cannot read");
	if (close(fd) < 0)
		err(1, "cannot close");
}

I do not have access to a Solaris or Mac OSX client to test, rick

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: David Noveck <[email-masked]>
Date: Sat, 25 Dec 2021 08:23:47 -0500

> and keep the discussion focused.

Good idea.  With a focused discussion, we could get this issue and possy
others you raise resolved in -05.

> POSIX-like systems) can function without the "owner-override".

I am prepared to make this a MAY if others agree and there is a clear
statement of what specifically this overide semantics consists of. I note,
however, that your statement which implies that such clients cannot
function with servers that do not implement this override makes me
skeptical.

> while I/O is done on the open_fd.

I was unaware of that.  I was recently looking at ontap's handling of this
(for a reason not connected with this document) and noted that the code
does read permission checks for reads using NFSv4 open files but not for
SMB/CIFS open files.  I presume that is because Microsoft specifies that
behavior.

>       I/O failures due for owner/owner_group/mode/ACL changes.

I don't have any problem with the spec saying that the server MAY allow IO
to proceed without permission checks if access was valid at the time of
open or MAY do so if the owner of the file is doing the IO.

>       and should remain specified,

It cannot "*remain* specified" (emphasis added) since it has never been
specified or defined.  That has to change.

even if it does appear to be in violation
>       of other parts of the RFCs.

We cannot have RFCs that contradict themselves.

> probably cause a lot of grief for extant NFSv4 clients on POSIX like

That could be part of a miscellaneous attributes I-D you were missing
about.  In any case, I intend to mention the possibility in the future
section of -05 once I have a clear definition of "owner override".

> If no one familiar with the Linux client answers this, I'll run a test on

I may wind up doing some testing in this area with Linux clients and will
report on what I see.

FWIW, I don't expect server choices in this area to discomfit the client
per se but don't expect that fact to be dispositive.  I think any grief
will be felt by users and applications and the spec should try not to
create problems for them.

> CAUTION: This email originated from outside of the University of Guelph.
> Do not click links or open attachments unless you recognize the sender and
> know the content is safe. If in doubt, forward suspicious emails to
> [email-masked]
>
>

---

From: David Noveck <[email-masked]>
Date: Sun, 26 Dec 2021 01:25:10 -0500

> "owner-override" was applied to Read/Write.

Is there documentation for this option?  I have googled for these words
several times and come up empty.  We need a clear definition of these
semantics to resolve this issue in -05 and that documentation might be
helpful.   If it isn't, could you point me to code, preferably in bsd,
implementing that option.

---

From: Rick Macklem <[email-masked]>
Date: Sun, 26 Dec 2021 16:45:39 +0000

David Noveck wrote:
> On Fri, Dec 24, 2021, 8:49 PM Rick Macklem <[email-masked]<mailto:[email-masked]>> wrote:
> > Rick Macklem wrote:
> > [stuff snipped]
> Is there documentation for this option?  I have googled for these words several 
> times and come up empty.  We need a clear definition of these semantics to 
> resolve this issue in -05 and that documentation might be helpful.   If it isn't, 
> could you point me to code, preferably in bsd, implementing that option.
Well, I'm not sure if/when the term "owner-override" cropped up, but here is
a description of it from RFC 1813 pg. 98:
   Another problem arises due to the usually stateful open
   operation.  Most operating systems check permission at open
   time, and then check that the file is open on each read and
   write request. With stateless servers, the server cannot
   detect that the file is open and must do permission checking
   on each read and write call. UNIX client semantics of access
   permission checking on open can be provided with the ACCESS
   procedure call in this revision, which allows a client to
   explicitly check access permissions without resorting to
   trying the operation. On a local file system, a user can open
   a file and then change the permissions so that no one is
   allowed to touch it, but will still be able to write to the
   file because it is open. On a remote file system, by
   contrast, the write would fail. To get around this problem,
   the server's permission checking algorithm should allow the
   owner of a file to access it regardless of the permission
   setting. This is needed in a practical NFS version 3 protocol
   server implementation, but it does depart from correct local
   file system semantics. This should not affect the return
   result of access permissions as returned by the ACCESS.

See the 3rd sentence from the end.

As far as I know, servers implement the same thing for Read/Write
for NFSv4. (FreeBSD does it for Read as well as Write, although the
above only refers to Write.)

For NFSv4 over encryption with client peer authentication, I think you
can argue the following:
- Since the client can be trusted.
and
- Encryption subverts "man in the middle attacks" and the ability to
  "sniff to acquire valid stateids".
that an appropriate Open/Lock/Delegation stateid is sufficient to
permission check the Read/Write and, as such, checking of mode/ACL
is not required.

--> Appropriate stateid would refer to a current one issued to that client
      where either the Open associated with the Open/Lock stateid includes
      the needed OPEN_SHARE_ACCESS or the Delegation is of the correct
      type (READ_DELEGATION or WRITE_DELEGATION).

The above *might* be preferred to "owner-override" for Read/Write
operations?

rick
ps: I am not sure if any NFSv4 server uses the stateid in Read/Write in
      the way described above.  At this time, FreeBSD only uses these
      stateids to check for conflicting Open/shares (byte range locks are
      always advisory).

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/param.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <err.h>
#include <unistd.h>

char buf[1024 * 1024];

int

________________________________________
From: nfsv4 <[email-masked]<mailto:[email-masked]>> on behalf of David Noveck <[email-masked]<mailto:[email-masked]>>
Sent: Friday, December 24, 2021 8:49 AM
To: NFSv4; nfsv4-chairs; [email-masked]<mailto:[email-masked]>
Subject: [nfsv4] Fwd: New Version Notification for draft-dnoveck-nfsv4-security-04.txt

CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to [email-masked]<mailto:[email-masked]>

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>>

Abstract:

The IETF Secretariat

_______________________________________________
nfsv4 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Rick Macklem <[email-masked]>
Date: Sun, 26 Dec 2021 17:10:56 +0000

Now it's my turn to ask for doc;-)
I've just started to look at the ACL section and I find
assorted references like:
/*
 * Calculate inherited ACL in a manner compatible with PSARC/2010/029.
 * It's also being used to calculate a trivial ACL, by inheriting from
 * a NULL ACL.
 */
in the FreeBSD code. (The ACL stuff was not done by me.)

I can find references to PSARC/2010/029 elsewhere, but have not located
the document. (I think it might be some Oracle doc?)

Anyone know where I can find it?
(Or is the contents of it now in RFC8881?)

Thanks for any info, rick

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat

---

From: David Noveck <[email-masked]>
Date: Tue, 28 Dec 2021 05:23:40 -0500

On Sun, Dec 26, 2021, 11:45 AM Rick Macklem <[email-masked]> wrote:

> several

Thanks.  This is helpful.

> for NFSv4.

>From your description and the testing you have done, it seems there is a
significant difference in that this special handling of the owner
permissions only applies to an open file in v4.  Is there a v4 server that
does this v3-like thing even for files that are not open?

(FreeBSD does it for Read as well as Write, although the
> above only refers to Write.)

I don't have a problem with that as long as:
 - it is limited to open files
 - It does not apply to the execute bit

> For NFSv4 over encryption with client peer authentication, I think you

Although I don't want to get into arguing in a standards-track document, I
think you have the essence of the matter right.

Since v4 is  stateful protocol, and now has reasonable security, we can
reasonably allow the server to rely on permission checking on open, if it
chooses.  Doing this only for the owner would also be covered.

>       type (READ_DELEGATION or WRITE_DELEGATION).

I will do something similar in -05.

>       stateids to check for conflicting Open/shares (byte range locks are

If that's the case, then, for v4, the it and uw bits would be ignored and
your test program would behave the same if you closed it and re-opened it
before the write.

Of that's the case, let me know so that I can address that issue for -05.

> From: nfsv4 <[email-masked]<mailto:[email-masked]>> on
> behalf of David Noveck <[email-masked]<mailto:[email-masked]>>
> [email-masked]<mailto:[email-masked]>
>
>
> From: <[email-masked]<mailto:[email-masked]><mailto:
> [email-masked]<mailto:[email-masked]>>>
> To: David Noveck <[email-masked]<mailto:[email-masked]
> ><mailto:[email-masked]<mailto:[email-masked]>>>
>
>
>

---

From: Rick Macklem <[email-masked]>
Date: Wed, 29 Dec 2021 04:32:10 +0000

David Noveck wrote:
> On Sun, Dec 26, 2021, 11:45 AM Rick Macklem wrote:
> > David Noveck wrote:
> > > Rick Macklem wrote:
> > > [stuff snipped]

> significant difference in that this special handling of the owner permissions only
> applies to an open file in v4.  Is there a v4 server that does this v3-like thing even
> for files that are not open?
The FreeBSD server does. It does the "owner-override" for Read/Write using
special stateids (which is what I assume you mean by "not open"?).
It only uses the stateid to check for a conflicting Open_Share_Deny. (For special
stateids, any Open_Share_Deny for the type of I/O would be considered
conflicting.

>  - it is limited to open files
I can see an argument for not doing "owner-override" for special stateids.
The counter argument would be along the lines of "nothing in the RFCs
says that it cannot be done for special stateids and defining that in this
draft changes the protocol.

To be honest, I am not aware of any extant clients that do Read/Write with
special stateids at this time.

> - It does not apply to the execute bit
Not sure why execute would be handled differently that Read?

> Although I don't want to get into arguing in a standards-track document,
> I think you have the essence of the matter right.
>
> Since v4 is  stateful protocol, and now has reasonable security, we can reasonably
> allow the server to rely on permission checking on open, if it chooses.  Doing this
> only for the owner would also be covered.
>
This was simply meant to say "I do not know if any extant server does what
I argued could be reasonable, above.

> >
> > If that's the case, then, for v4, the it and uw bits would be ignored and your test
> > program would behave the same if you closed it and re-opened it before the
> > write.
Not sure what you mean? The permission checking would not change for Opens.
If done locally on the client, due to a delegation being issued, an Access operation
would be used.

rick

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/param.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <err.h>
#include <unistd.h>

char buf[1024 * 1024];

int

________________________________________
From: nfsv4 <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> on behalf of David Noveck <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>>
Sent: Friday, December 24, 2021 8:49 AM
To: NFSv4; nfsv4-chairs; [email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>
Subject: [nfsv4] Fwd: New Version Notification for draft-dnoveck-nfsv4-security-04.txt

CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to [email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>><mailto:[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>><mailto:[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>>>

Abstract:

The IETF Secretariat

_______________________________________________
nfsv4 mailing list
[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Rick Macklem <[email-masked]>
Date: Wed, 29 Dec 2021 17:42:09 +0000

Here are specific comments related to ACE4_APPEND_DATA:

>From the draft:
The action of modifying a file's data, but only starting at
         EOF.  This allows for the specification of append-only files,
         by allowing ACE4_APPEND_DATA and denying ACE4_WRITE_DATA to the
         same user or group.

         [Consensus Needed (Item #8a)]: As there is no way for the
         server to decide, in processing an OPEN or ACCESS request,
         whether subsequent WRITEs will extend the file or not, the
         server will treat masks containing only WRITE_DATA, only
         APPEND_DATA or both, identically.

I believe it would be preferable to say "...the server will ignore
APPEND_DATA and will only enforce WRITE_DATA".
Why?
Well, my understanding is that APPEND_DATA does not apply to
any old Write that happens to start at EOF, but specifically "append-only"
file operations. (A POSIX write done on a file opened with the O_APPEND
flag would be one example.)
Since "append-only" file operations do not exist in NFSv4 (as you noted),
then ignoring APPEND_DATA seems more correct that assuming it to be
equivalent to WRITE_DATA.

For example, assume a file with APPEND_DATA allowed, but WRITE_DATA not
allowed that is of size == 0. If APPEND_DATA is sufficient to allow an Open with
OPEN_SHARE_ACCESS_WRITE, then a client could do the following:
- Open the file with OPEN_SHARE_ACCESS_WRITE
  - Write data to the file sequentially (the most common write usage).
--> Not what APPEND_DATA allowed but WRITE_DATA not allowed means,
       from my understanding.

Then there is this in the draft:
  *  Servers that do not distinguish between WRITE_DATA and APPEND_DATA
      need to make it clear to clients that support for append-only
      files is not present.  To do that, requests to set NFSv4 ACLs
      where the handling for these two masks are different for any
      specified user or group are to be rejected with
      NFS4ERR_ATTRNOTSUPP.

First, given that there is no OPEN_SHARE_ACCESS_APPEND, this applies to
all NFSv4 servers.

Second, although I can understand that this would "make it clear to clients",
there are issues with doing this for systems like FreeBSD, where NFSv4 ACLs
are used locally as well as for the NFSv4 server.
For example, on FreeBSD:
- A setfacl(1) can set any combination of APPEND_DATA, WRITE_DATA or both
  APPEND_DATA and WRITE_DATA for allow/deny when done locally on UFS or ZFS
  file systems.
- A getfacl(1) will show them separately.
If FreeBSD were to enforce the above in the NFSv4 server, the getfacl(1) on a FreeBSD
NFSv4 client would show any combination as above, but the setfacl(1) would fail
for the case where only one of APPEND_DATA/WRITE_DATA is being set.
Users would find this a weird, irritating inconsistency. (As such, it won't be the case
for FreeBSD NFSv4 servers.)

I'd prefer that setting any combination be allowed, but that enforcement of APPEND_DATA
not be done for the NFSv4 server and that APPEND_DATA be handled like other non-enforcible
bits, such as SYNCHRONIZE.

rick

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Fri, Dec 24, 2021 at 8:31 AM
Subject: New Version Notification for draft-dnoveck-nfsv4-security-04.txt
To: David Noveck <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat