Subject: [nfsv4] pseudo-fs traversal with wrong RPCSEC_GSS service

From: Saul Tamari <[email-masked]>
Date: Thu, 18 Apr 2013 12:39:09 -0400

Hi,

I have a question regarding the expected behavior from an NFSv4 server
when it exports some directory that requires a certain security
flavor.
For example lets say that a given server exports one path /a/b and
requires all accesses to b to use krb5p.

Should this server fail any request that does not use RPCSEC_GSS with
privacy service (probably with NFS4ERR_WRONGSEC) ?
Is it ok to allow PUTROOTFH, LOOKUP and GETATTR for the pseudefs
entries even without the right kind of security service (e.g.
authentication) in place?

Thanks,
Saul

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 18 Apr 2013 16:58:39 +0000

> privacy service (probably with NFS4ERR_WRONGSEC) ?

Yes.

> authentication) in place?

Clients are supposed to have strategies for autonegotiating security in
the above situation, however you may find that some older clients are
less capable than the spec assumes. At least that is the case for older
Linux clients...

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 10:24:45 -0700

> privacy service (probably with NFS4ERR_WRONGSEC) ?

It's subtle.

The initial SETCLIENTID/SETCLIENTID_CONFIRM is done with a machine credential, ideally.  But export options generally don't determine what flavors a server requires for establishing a client ID.

I think clients should be free to use any flavor when establishing their client ID.  They should prefer strong security, of course, but I don't think the server should require it, and I'm not aware of server administrative interfaces that allow limiting flavors in this way.

Thus, there are some requests that a server may accept without requiring privacy.

> authentication) in place?

Server implementations I'm familiar with take a union of flavors specified on their exports, and use that as the set of flavors required for accessing the pseudo-fs.  There is no requirement to do this, however.

I think the protocol spec suggests that the pseudo-fs SHOULD be world-readable, in which case, the server may allow AUTH_NULL or AUTH_UNIX access to its pseudo-fs even if strong security is required for its exports.

There is no requirement for servers to support AUTH_UNIX (only AUTH_NULL and AUTH_GSS flavors are required to be supported).  In practice, every server I'm aware of supports AUTH_UNIX.

Clients typically try PUTROOTFH first with AUTH_UNIX if no "sec=" is specified on the mount command line.  A scalable server might always allow AUTH_UNIX for pseudo-fs access, to avoid the WRONGSEC dance with every client during a mount flood.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: Saul Tamari <[email-masked]>
Date: Thu, 18 Apr 2013 15:14:41 -0400

On Thu, Apr 18, 2013 at 1:24 PM, Chuck Lever <[email-masked]> wrote:
[snip]

> Thus, there are some requests that a server may accept without requiring privacy.

I couldn't find this documented in RFC3530.
Did I miss is or its not documented?
Is there a document similar to RFC 2623 (NFS Version 2 and Version 3
Security Issues and the NFS Protocol's Use of RPCSEC_GSS and Kerberos
V5) for NFSv4?

> I think the protocol spec suggests that the pseudo-fs SHOULD be world-readable, in which case, the server may allow AUTH_NULL or AUTH_UNIX access to its pseudo-fs even if strong security is require…

I wasn't able to find such suggestion in RFC 3530 but I might have missed it.

[snip]
> Clients typically try PUTROOTFH first with AUTH_UNIX if no "sec=" is specified on the mount command line.  A scalable server might always allow AUTH_UNIX for pseudo-fs access, to avoid the WRONGSEC…

Thanks for the good advice.

Saul

---

From: Andy Adamson <[email-masked]>
Date: Thu, 18 Apr 2013 15:17:18 -0400

> > privacy service (probably with NFS4ERR_WRONGSEC) ?

Yes it should fail any request to the subtree under "b" that does not use
RPCSEC_GSS with krb5p. But /a could have a different security requirement:
for example, /a could be the NFSv4 Pseudo file system and allow AUTH_SYS
access, and krb5p would then be required to traverse into /a/b.

--Andy

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Saul Tamari <[email-masked]>
Date: Thu, 18 Apr 2013 15:18:17 -0400

On Thu, Apr 18, 2013 at 12:58 PM, Myklebust, Trond
<[email-masked]> wrote:
[snip]
> less capable than the spec assumes. At least that is the case for older

I noticed that when I added support for RPCSEC_GSS to our server so I
allowed permissive traversal of the pseudo-fs.
I'm now trying to see if my implementation is legit from the RFC standpoint.

Thanks,
Saul

---

From: Saul Tamari <[email-masked]>
Date: Thu, 18 Apr 2013 15:24:32 -0400

On Thu, Apr 18, 2013 at 3:17 PM, Andy Adamson <[email-masked]> wrote:
[snip]
> access, and krb5p would then be required to traverse into /a/b.

Is this case also true when the only export in the system is for /a/b
(with krb5p) ?
Would you suggest that /a would be readable for clients using AUTH_SYS ?

Thanks,
Saul

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 12:37:54 -0700

> Did I miss is or its not documented?

This is not documented.

The key is that SETCLIENTID and SETCLIENTID_CONFIRM be performed with some flavor and a machine credential.  The next operation a client does is typically PUTROOTFH, but it does not necessarily use the same flavor or credential.

> I wasn't able to find such suggestion in RFC 3530 but I might have missed it.

Chapter 8 of RFC 3530bis discusses pseudo-fs security considerations, and an expanded discussion of the topic appears in RFC 5661 section 7.8.

I don't remember where I saw the world-readable suggestion.

> Thanks for the good advice.

I encourage you to review section 15.33.5 of RFC 3530bis to understand the limitations of security negotiation when accessing the pseudo-fs with NFSv4.0.

---

From: Rick Macklem <[email-masked]>
Date: Thu, 18 Apr 2013 16:15:51 -0400 (EDT)

Chuck Lever wrote:
> >> The initial SETCLIENTID/SETCLIENTID_CONFIRM is done with a machine
> >> credential, ideally. But export options generally don't determine
> >> what flavors a server requires for establishing a client ID.
> >>
The FreeBSD NFSv4 server does have a line in /etc/exports that specifies
where the NFSv4 root dir is and what security flavours may be used for
operations not associated with a cfh, such as SetClientID and SetClientIDConfirm.

> >> I think clients should be free to use any flavor when establishing
> >> their client ID. They should prefer strong security, of course, but
> >> I don't think the server should require it, and I'm not aware of
> >> server administrative interfaces that allow limiting flavors in
> >> this way.
> >>
I haven't seen any problems with restricting it, as above. However, I
think it does need to be sysadmin configurable.

rick

> The key is that SETCLIENTID and SETCLIENTID_CONFIRM be performed with
> some flavor and a machine credential. The next operation a client does
> is typically PUTROOTFH, but it does not necessarily use the same
> flavor or credential.
> 
> >> Server implementations I'm familiar with take a union of flavors
> >> specified on their exports, and use that as the set of flavors
> >> required for accessing the pseudo-fs. There is no requirement to do
> >> this, however.
> >>
> >> I think the protocol spec suggests that the pseudo-fs SHOULD be
> >> world-readable, in which case, the server may allow AUTH_NULL or
> >> AUTH_UNIX access to its pseudo-fs even if strong security is
> >> required for its exports.
> >
> Chapter 8 of RFC 3530bis discusses pseudo-fs security considerations,
> and an expanded discussion of the topic appears in RFC 5661 section
> 7.8.
> 
> >> There is no requirement for servers to support AUTH_UNIX (only
> >> AUTH_NULL and AUTH_GSS flavors are required to be supported). In
> >> practice, every server I'm aware of supports AUTH_UNIX.
> >>
> >> Clients typically try PUTROOTFH first with AUTH_UNIX if no "sec="
> >> is specified on the mount command line. A scalable server might
> >> always allow AUTH_UNIX for pseudo-fs access, to avoid the WRONGSEC
> >> dance with every client during a mount flood.
> >
> I encourage you to review section 15.33.5 of RFC 3530bis to understand
> the limitations of security negotiation when accessing the pseudo-fs
> with NFSv4.0.
> 
> --
> Chuck Lever

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 13:36:49 -0700

> operations not associated with a cfh, such as SetClientID and SetClientIDConfirm.

The Linux client tries SETCLIENTID with krb5i first, then with AUTH_NULL if that fails.  When we tested this at Connectathon, we did not find a server where this logic did not work.

> think it does need to be sysadmin configurable.

Why restrict SETCLIENTID?

---

From: Rick Macklem <[email-masked]>
Date: Thu, 18 Apr 2013 16:56:58 -0400 (EDT)

Chuck Lever wrote:
> The Linux client tries SETCLIENTID with krb5i first, then with
> AUTH_NULL if that fails. When we tested this at Connectathon, we did
> not find a server where this logic did not work.
> 
Well, the FreeBSD server can be configured so that the above will not
work. I can do that at the June bakeathon, if you'd like.
(Most folks just enable auth_sys, from what I've seen w.r.t. FreeBSD.)

rick

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 14:02:37 -0700

> (Most folks just enable auth_sys, from what I've seen w.r.t. FreeBSD.)

For a machine cred, AUTH_NULL and AUTH_UNIX provide the same level of security.  And, servers aren't required to support AUTH_UNIX.  So we picked AUTH_NULL as the fallback.  It's a one-line fix if AUTH_UNIX turns out to be a better choice.

I don't understand why SETCLIENTID needs to be restricted at all, though.  Is there a particular use case where a system administrator needs to restrict the flavor used to issue SETCLIENTID?

---

From: Rick Macklem <[email-masked]>
Date: Thu, 18 Apr 2013 17:26:03 -0400 (EDT)

Chuck Lever wrote:
> For a machine cred, AUTH_NULL and AUTH_UNIX provide the same level of
> security. And, servers aren't required to support AUTH_UNIX. So we
> picked AUTH_NULL as the fallback. It's a one-line fix if AUTH_UNIX
> turns out to be a better choice.
> 
Well, you use the term "machine credential". The RFC refers to
"principal", which to me is a name (as in kerberos principal name).
(I would think of machine credentials as a gssapi host principal like
 "[email-masked]" in a keytab entry, but users are also allowed
 to do kerberized mounts, where the principal would be a user principal
 like "rick". The FreeBSD client allows both and the Mac client only
 does the latter, from what I recall.)
For AUTH_SYS, FreeBSD just uses the uid number as the principal name.
(I have no idea what principal AUTH_NULL is, but if you assumed uid 0,
 I can understand that.)

I would agree that krb5p doesn't really gain anything w.r.t. security
over krb5i, but it happens that FreeBSD just parses a list of security
flavours, so it is possible to configure such a case.

> I don't understand why SETCLIENTID needs to be restricted at all,
> though. Is there a particular use case where a system administrator
> needs to restrict the flavor used to issue SETCLIENTID?
> 
Well, if a malicious person sends SETCLIENTID/SETCLIENTIDCONFIRM to
a server that messes things up pretty quickly. (Just because the client
machine is trusted enough to mount the volume doesn't imply that
all users logged into that machine can be trusted to not write programs
that send SETCLIENTID to the server.)

Consider this as the v4.0 equivalent to the kind of attack that SSV is
supposed to protect against.

rick

> >>> I haven't seen any problems with restricting it, as above.
> >>> However,
> >>> I
> >>>>> Is there a document similar to RFC 2623 (NFS Version 2 and
> >>>>> Version
> >>>>> 3
> >>>>>> required for accessing the pseudo-fs. There is no requirement
> >>>>>> to
> >>>>>> do
> >>>>>> this, however.
> >>>>>>
> >>>>> I wasn't able to find such suggestion in RFC 3530 but I might
> >>>>> have
> >>>>> missed it.
> >>>>

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 15:12:48 -0700

> that send SETCLIENTID to the server.)

Just above you argue that users should be allowed to perform a SETCLIENTID in order to perform a mount.  Here you say that users should not be allowed to perform SETCLIENTID, which is done before a mount, because that's a security risk.  That seems inconsistent.

The reason why only machine credentials should be used is that a client needs to use the same principal every time it performs a SETCLIENTID, otherwise the server rejects the attempt to establish a fresh client ID with NFS4ERR_CLID_INUSE.

If you use a user credential for SETCLIENTID and the user logs out and destroys her credential, the server won't allow a SETCLIENTID again with that nfs_client_id4 until the old lease expires.

If a machine credential is used, the same credential can be used every time a client ID must be re-established.  The server will reject any attempt to establish a client ID using a user credential while that lease continues to exist.  In other words, a machine credential is always available and essentially never expires.

Thus, using a machine credential (with AUTH_GSS) locks out attempts by users to purge the lease on the server.  And, a user can establish a mount, leave the client, and a new user can come along, and the lease is still protected and can be re-established automatically.

If a client always tries to use krb5i first for SETCLIENTID, then we get better security, even for AUTH_UNIX mounts (as long as the client and server support Kerberos, of course).  Falling back to AUTH_NULL means a client can still mount (and a user credential is still required, though it's not used for SETCLIENTID).  There is no security using AUTH_UNIX or AUTH_NULL anyway.

> supposed to protect against.

Your argument is in favor of using a strong security flavor for SETCLIENTID, which I agree with.

Even though it's a bad idea (and maybe even insecure) to use a user credential for SETCLIENTID, servers generally don't prevent their use for SETCLIENTID.  It seems similarly limited to restrict the security flavors allowed for SETCLIENTID.

---

From: Rick Macklem <[email-masked]>
Date: Thu, 18 Apr 2013 18:56:56 -0400 (EDT)

Chuck Lever wrote:
> >>>>> where the NFSv4 root dir is and what security flavours may be
> >>>>> used
> >>>>> for
> Just above you argue that users should be allowed to perform a
> SETCLIENTID in order to perform a mount. Here you say that users
> should not be allowed to perform SETCLIENTID, which is done before a
> mount, because that's a security risk. That seems inconsistent.
> 
Yes, I didn't state the above correctly.

What I was thinking of is where the user doesn't have a kerberos
principal, but uses AUTH_UNIX or AUTH_NULL. That is the case I
was interested in restricting.

I suppose that allowing "user principals" should be optional,
but I'll admit that the FreeBSD server doesn't have that optional restriction.
The FreeBSD client doesn't allow non-root mounts by default and that
has to be enabled by root before it can be done.

> The reason why only machine credentials should be used is that a
> client needs to use the same principal every time it performs a
> SETCLIENTID, otherwise the server rejects the attempt to establish a
> fresh client ID with NFS4ERR_CLID_INUSE.
> 
> If you use a user credential for SETCLIENTID and the user logs out and
> destroys her credential, the server won't allow a SETCLIENTID again
> with that nfs_client_id4 until the old lease expires.
> 
Yes, the use of a user principal is for the case where a given user
does the mount and dismounts before/when they log out. The intent is
for cases like laptops, where the user wants to do a short term mount
to a server to access their files.
Mac OSX only implements this kind of Kerberized NFS mounts from what I
recall, but I haven't looked at their current NFSv4 client.

> If a machine credential is used, the same credential can be used every
> time a client ID must be re-established. The server will reject any
> attempt to establish a client ID using a user credential while that
> lease continues to exist. In other words, a machine credential is
> always available and essentially never expires.
> 
The hassle with what you call a machine credential (I'm assuming a host
based principal in a keytab file) is that they must be generated for
each machine and securely transferred to the client. Not too bad for
rack mounted boxes in a machine room, but a big bother for 10,000 students
with various mobile devices (for example).

One of the original goals was "Improved access and good performance on the Internet"
and this is a lot easier to implement via "user principals" for SETCLIENTID.
(The FreeBSD server does save the principal name in the clientid structure,
 so it can check for "same principal".)

> Thus, using a machine credential (with AUTH_GSS) locks out attempts by
> users to purge the lease on the server. And, a user can establish a
> mount, leave the client, and a new user can come along, and the lease
> is still protected and can be re-established automatically.
> 
> If a client always tries to use krb5i first for SETCLIENTID, then we
> get better security, even for AUTH_UNIX mounts (as long as the client
> and server support Kerberos, of course). Falling back to AUTH_NULL
> means a client can still mount (and a user credential is still
> required, though it's not used for SETCLIENTID). There is no security
> using AUTH_UNIX or AUTH_NULL anyway.
> 
> Even though it's a bad idea (and maybe even insecure) to use a user
> credential for SETCLIENTID, servers generally don't prevent their use
> for SETCLIENTID. It seems similarly limited to restrict the security
> flavors allowed for SETCLIENTID.
> 
I think that a user with a valid Kerberos user principal credential
(valid TGT) is more trusted than a client using AUTH_SYS with uid==0,
but all the FreeBSD server does is try to allow a sysadmin to restrict
things in whatever way they feel is appropriate.

rick

> >>>>>>>> their client ID. They should prefer strong security, of
> >>>>>>>> course,
> >>>>>>>> but
> >>>>>>>> Thus, there are some requests that a server may accept
> >>>>>>>> without
> >>>>>>>> requiring privacy.
> >>>>>> The key is that SETCLIENTID and SETCLIENTID_CONFIRM be
> >>>>>> performed
> >>>>>> with
> >>>>>> some flavor and a machine credential. The next operation a
> >>>>>> client
> >>>>>> does

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Apr 2013 16:10:05 -0700

> with various mobile devices (for example).

Some enterprises want to deploy a large number of multi-user clients, each without a keytab.  In that case, allowing SETCLIENTID with AUTH_NULL prevents the problem of having to re-establish a SETCLIENTID when the original user credential has wandered off to lunch.

---

From: Rick Macklem <[email-masked]>
Date: Fri, 19 Apr 2013 10:42:15 -0400 (EDT)

Chuck Lever wrote:
> >>>>>> On Apr 18, 2013, at 1:15 PM, Rick Macklem
> >>>>>> <[email-masked]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Chuck Lever wrote:
> >>>>>> AUTH_NULL if that fails. When we tested this at Connectathon,
> >>>>>> we
> >>>>>> did
> >>>>> Well, the FreeBSD server can be configured so that the above
> >>>>> will
> >>>>> not
> >>>> For a machine cred, AUTH_NULL and AUTH_UNIX provide the same
> >>>> level
> >>>> of
> >>> (I have no idea what principal AUTH_NULL is, but if you assumed
> >>> uid
> >>> 0,
> >>> I can understand that.)
> >>>
> Some enterprises want to deploy a large number of multi-user clients,
> each without a keytab. In that case, allowing SETCLIENTID with
> AUTH_NULL prevents the problem of having to re-establish a SETCLIENTID
> when the original user credential has wandered off to lunch.
> 
Just fyi, the FreeBSD server won't handle any compound RPC with AUTH_NULL.
This was how it was implemented for NFSv3 and I didn't change it for NFSv4.
(ie. Only the Null RPC can have AUTH_NULL.)

Also, for want of anything better, it uses the "uid" as the principal name
for a SETCLIENTID done over AUTH_SYS.

Although I won't argue it is more secure, for the sake of interoperability,
using AUTH_SYS instead of AUTH_NULL for this case would be better, because
AUTH_NULL isn't going to work for FreeBSD servers.

rick

> >>> Consider this as the v4.0 equivalent to the kind of attack that
> >>> SSV
> >>> is
> >>>>>>>>>> I don't think the server should require it, and I'm not
> >>>>>>>>>> aware
> >>>>>>>>>> of
> >>>>>>>>>> server administrative interfaces that allow limiting
> >>>>>>>>>> flavors
> >>>>>>>>>> in
> >>>>>>>>>> this way.
> >>>>>>>>>>
> >>>>>>>>>> I think the protocol spec suggests that the pseudo-fs
> >>>>>>>>>> SHOULD
> >>>>>>>>>> be
> >>>>>>>>>> world-readable, in which case, the server may allow
> >>>>>>>>>> AUTH_NULL
> >>>>>>>>>> or
> >>>>>>>>>> AUTH_NULL and AUTH_GSS flavors are required to be
> >>>>>>>>>> supported).
> >>>>>>>>>> In

---

From: Chuck Lever <[email-masked]>
Date: Fri, 19 Apr 2013 09:00:48 -0700

> (ie. Only the Null RPC can have AUTH_NULL.)

Thanks for pointing that out!

> for a SETCLIENTID done over AUTH_SYS.

Using AUTH_SYS for SETCLIENTID was what Linux used to do, and I believe it did use UID 0 in this case.

> AUTH_NULL isn't going to work for FreeBSD servers.

I think we can make that change in Linux.

This is an interoperability problem that arises from the protocol specification, unfortunately.  RFC 3530bis section 15.33.5 has this:

>    o  For LOOKUPP, PUTROOTFH and PUTPUBFH, the client will be unable to
>       use the SECINFO operation since SECINFO requires a current
>       filehandle and none exist for these two operations.  Therefore,
>       the client must iterate through the security triples available at
>       the client and reattempt the PUTROOTFH or PUTPUBFH operation.  In
>       the unfortunate event none of the MANDATORY security triples are
>       supported by the client and server, the client SHOULD try using
>       others that support integrity.  Failing that, the client can try
>       using AUTH_NONE, but because such forms lack integrity checks,
>       this puts the client at risk.  Nonetheless, the server SHOULD
>       allow the client to use whatever security form the client requests
>       and the server supports, since the risks of doing so are on the
>       client.

It recommends using AUTH_NONE here, and not AUTH_SYS.  Thus a client that implements this suggestion would not interoperate with a server that does not support AUTH_NONE and does not deploy RPCGSS.

This recommendation is problematic given section 3.2:

>       Other flavors, such as, AUTH_NONE, AUTH_SYS, and AUTH_DH MAY be
>       implemented as well.

If AUTH_NONE is optional, as is AUTH_SYS, then a client can't expect to have a 100% chance to interoperate in this case.

So either AUTH_SYS and AUTH_NONE are optional, or they are not.  The protocol specification should not, IMO, make a recommendation like this if one of the recommended-to-try flavors is not guaranteed to be supported.  When I read this passage months ago, I had assumed consistency between 3.2 and 15.33.5.  The recommendation is describing hunting over what I thought were all mandatory-to-implement flavors.  Why explicitly skip AUTH_SYS here?

Both 3530 and 5661 treat AUTH_NONE as optional, but 3530 "relies on" having AUTH_NONE to handle this particular use case.  Perhaps 3530bis should be updated to make AUTH_NONE or AUTH_SYS required-to-implement, or 15.33.5 could be updated somehow (no ideas here)?