Subject: [nfsv4] Review of draft-ietf-nfsv4-integrity-measurment-01

From: David Noveck <[email-masked]>
Date: Thu, 27 Sep 2018 19:52:47 -0400

*General Comments*
The document right now does not have sufficient information to allow
actual, working implementations to be developed, but it is a good start on
an eventual document and should allow some limited prototypeing to be done,
while the process of filling out necessary information proceeds:

The things that we are waiting for are:

   - A definition of the fornat of the provenance information.
   - Further clarification about how attribute verification is to be done.
   - A specification of the security policy that servers are to apply.

This leads to REQUIREMENTS that it are impossible to comply with and I've
noted instances of that below.

Another troublesome issue is the treatment of IMA.  Some parts of the
document present IMA as a one possile use among many for provenance
information, while the reality is that it is highly unlikely for there to
be more than one.  I'm not sure this issue is resolvable.

*Comments by Section*

*4.1. **XDR Definition*

In the first paragraph suggest deleting the word "GETATTR".

Suggest the following replacement for the second paragraph:

Within NFSv4, file provenance information is treated as opaque data which
need not be interpreted by the server. In order for multiple clients to
co-operate in generating and verifying file provenance information, there
needs to be a definition of the format in which the information is to be
stored, preferably in a self-describing format. This document does not
desscribe that format, requiring that clients use other means to ensure
consistent use of this opaque data.

Suggest the following replacement for the third paragraph:

In order to enable a single file provenance information payload to be
retrieved or updated via a single RPC, and to constrain the transport
resources required for the operations defined here, the length of the
file provenance information is strictly limited.  It MUST  NOT exceed
4096 bytes in length.

*4.2.  Storing File Provenance Metadata*

With regard to the second paragraph, I'll just note here that the
phrase "credential that is unauthorized to replace the
FATTR4_FILE_PROVENANCE attribute" appears, leaving the implementer in
doubt about when he "MUST return NFS4ERR_ACCESS".   I'll make
suggestions on dealing witth this issue when we review *5.  Security
Considerations.*

With regard to the third paragraph, NFS4ERR_NAMETOOLONG  seems wrong
since file provenance information is not a name.  I believe
NFS4ERR_INVAL is a better fit.

The fourth paragraph as written does not deal properly with the case
in which some filesystems on the server support file provenance
information and other do not.   It indicates that NFS4ERR_type is to
be returned in this case, when NFS4ERR_ATTRNOTSUPP should be returned.
Suggest the following replacement:

When a SETATTR is presented to an NFS version 4.2 server that supports
FATTR4_FILE_PROVENANCE and the target object is in a file system which
supports FATTR4_FILE_PROVENANCE but object does not support this
attribute, the server MUST respond with NFS4ERR_WRONGTYPE.  In the
case in which the file system does not support this  attribute, the
error NFS4ERR_ATTRNOTSUPP is returned.

*4.3. Retrieving File Provenance Metadata *

The second paragraph has an issue similar to that in the fourth
paraphaph of *4.2.
Storing File Provenance Metadata*. Suggest the following replacement:

When a GETATTR is presented to an NFS version 4.2 server that supports
FATTR4_FILE_PROVENANCE and the target object is in a file system which
supports FATTR4_FILE_PROVENANCE but object does not support this
attribute, the server MUST respond with NFS4ERR_WRONGTYPE.  In the
case in which the file system does not support this  attribute, the
attribute bit is not set in the response and no error is returned.

The third paragraph doesn't make sense to me.   I don't really see the
point of saying the server "MUST NOT" do something, that he has no
possible way to do.   He doesn't understand the format of the
provenance data so it basically impossible for him to do a provenance
verification.  Saying he MUST NOT prevent access based on verification
results is confusing since there is no point in saying you MUST NOT do
something you cannot do.  Suggest the following replacement:

The server has no role (and generally no ability) to verify provenance
information.   As a result, if provenance veification is to be done,
it must be done by the client, who needs acces to the file data and
provenance information provided by the server, who is charged with
providing each of these to the client independently.

*5.  Security Considerations*

The first paragraph states a REQUIREMENT on the server with no way for the
server implementor to figure out what he is supposed to do to conform to
the REQUIREMENT.   I don't see the point of doing this.   Since the actual
security requirements are not known now,  it is best to indicate that and
wait until the requirements are known to put in the RFC2119 language once
they are.  Alternatively, you could put in some preliminary requirements
that could be used for prototyping and indicate that they are subject to
change.

I think it would be better to simply forbid use of non-cryptographically
signed provenance informattion, rather than make speccial provisions for
these "rare" cases as is done in the third parapraph.   We are pretty sure
that the provenance information for IMA will be cryptograpihically signed
and it is just barely possible that another use will be found.   I don't
see the point of making specual provision for the (just about useless) case
in which that new use is not cryptographically signed.

---

From: Chuck Lever <[email-masked]>
Date: Fri, 28 Sep 2018 09:19:23 -0700

Hi Dave-

Thanks for the comments. I'll integrate these into the next
revision, along with Craig's and Bruce's comments.

Some further remarks and clarifications appear below.

> On Sep 27, 2018, at 4:52 PM, David Noveck <[email-masked]> wrote:
> 
> General Comments
> The document right now does not have sufficient information to allow actual, working implementations to be developed, but it is a good start on an eventual document and should allow some limited prototypeing to be done, while the process of filling out necessary information proceeds:  
> 
> 	• A definition of the fornat of the provenance information.

I don't expect the draft to ever provide a format for the provenance
information. This information is _opaque_ to NFS, and at least in IMA's
case, there is no published normative specification that defines it.
There is no practical way to specify the format in this document.

I thought the point of this revision was to make that clear, but
reviewers have been confused on this point. What more can the draft
do to make it clear?

> 	• Further clarification about how attribute verification is to be done.

That will be left to another document. This is a separate use case,
possibly not as well defined as the IMA use case at this time, and
therefore it will have to wait for the local implementation to
settle down.

There are two main issues to resolve:
- not all file attributes protected by EVM are exposed by NFS
- the attributes that are exposed might have different formats
on different client OSes, which would generate different hash
results

> 	• A specification of the security policy that servers are to apply.

I plan to include some guidelines and examples, but actual formal
policy will be left up to implementers.

> This leads to REQUIREMENTS that it are impossible to comply with and I've noted instances of that below.
> 
> Another troublesome issue is the treatment of IMA.  Some parts of the document present IMA as a one possile use among many for provenance information, while the reality is that it is highly unlikely for there to be more than one.  I'm not sure this issue is resolvable.  
> 
> Comments by Section
> 
> 4.1. XDR Definition
> 
> Within NFSv4, file provenance information is treated as opaque data which need not be interpreted by the server.  In order for multiple clients to co-operate in generating and verifying file provenance information, there needs to be a definition of the format in which the information is to be stored, preferably in a self-describing format.  This document does not desscribe that format, requiring that clients use other means to ensure consistent use of this opaque data.
> 
> In order to enable a single file provenance information payload to be retrieved or updated via a single RPC, and to constrain the transport resources required for the operations defined here, the length of the file provenance information is strictly limited.  It MUST  NOT exceed 4096 bytes in length.
> 4.2.  Storing File Provenance Metadata
> With regard to the second paragraph, I'll just note here that the phrase "credential that is unauthorized to replace the FATTR4_FILE_PROVENANCE attribute" appears, leaving the implementer in  doubt about when he "MUST return NFS4ERR_ACCESS".   I'll make suggestions on dealing witth this issue when we review 5.  Security Considerations. 
> With regard to the third paragraph, NFS4ERR_NAMETOOLONG  seems wrong since file provenance information is not a name.  I believe NFS4ERR_INVAL is a better fit.
> The fourth paragraph as written does not deal properly with the case in which some filesystems on the server support file provenance information and other do not.   It indicates that NFS4ERR_type is to be returned in this case, when NFS4ERR_ATTRNOTSUPP should be returned.  Suggest the following replacement:
> When a SETATTR is presented to an NFS version 4.2 server that supports FATTR4_FILE_PROVENANCE and the target object is in a file system which supports FATTR4_FILE_PROVENANCE but object does not support this attribute, the server MUST respond with NFS4ERR_WRONGTYPE.  In the case in which the file system does not support this  attribute, the error NFS4ERR_ATTRNOTSUPP is returned. 
> 4.3.  Retrieving File Provenance Metadata 
> 
> The second paragraph has an issue similar to that in the fourth paraphaph of 4.2.  Storing File Provenance Metadata.  Suggest the  following replacement:
> When a GETATTR is presented to an NFS version 4.2 server that supports FATTR4_FILE_PROVENANCE and the target object is in a file system which supports FATTR4_FILE_PROVENANCE but object does not support this attribute, the server MUST respond with NFS4ERR_WRONGTYPE.  In the case in which the file system does not support this  attribute, the attribute bit is not set in the response and no error is returned.
> The third paragraph doesn't make sense to me.   I don't really see the point of saying the server "MUST NOT" do something, that he has no possible way to do.   He doesn't understand the format of the provenance data so it basically impossible for him to do a provenance verification.  Saying he MUST NOT prevent access based on verification results is confusing since there is no point in saying you MUST NOT do something you cannot do.  Suggest the following replacement:
> The server has no role (and generally no ability) to verify provenance information.   As a result, if provenance veification is to be done, it must be done by the client, who needs acces to the file data and provenance information provided by the server, who is charged with providing each of these to the client independently.

A provenance assessor on the NFS server can limit access to
protected files by users on that server. But not all NFS
servers have a user execution environment. Those that do not
will be unlikely to have a provenance assessment capability.

For users on clients, a client's local provenance assessor must
make the determination if a protected file is safe to use. Thus
an NFS server that has a provenance assessment capability must
not use it to deny access to protected files.

Is that better?

> 5.  Security Considerations
> The first paragraph states a REQUIREMENT on the server with no way for the server implementor to figure out what he is supposed to do to conform to the REQUIREMENT.   I don't see the point of doing this.

I'm confused by this remark. The nfsv4 WG has a history of using
RFC 2119 terms in the Security Considerations section in this way.

"Don't (DO NOT) allow unprivileged principals to alter the FPI."
seems pretty straightforward to me. Do we need to state what
constitutes an acceptable authority level?

> Since the actual security requirements are not known now,  it is best to indicate that and wait until the requirements are known to put in the RFC2119 language once they are.  Alternatively, you could put in some preliminary requirements that could be used for prototyping and indicate that they are subject to change.

I agree that this section is currently "not ready for publication"
but I don't see the harm in leaving this general requirement as a
dog-ear for something that needs more work.

> I think it would be better to simply forbid use of non-cryptographically signed provenance informattion, rather than make speccial provisions for these "rare" cases as is done in the third parapraph.   We are pretty sure that the provenance information for IMA will be cryptograpihically signed and it is just barely possible that another use will be found.   I don't see the point of making specual provision for the (just about useless) case in which that new use is not cryptographically signed.   

--
Chuck Lever

---

From: "Everhart, Craig" <[email-masked]>
Date: Fri, 28 Sep 2018 16:41:20 +0000

I, for one, have big trouble with a putative NFS specification that imposes semantics on NFS implementations to which they cannot adhere, or to which they don't have enough information to know how to adhere.  It's mysterious enough that the proposed attribute has opaque semantics; that this draft will not "ever provide a format for the provenance information", expecting that NFS will provide a transparent communication channel for some feature.  But that's apparently not all.  Indeed, NFS clients/servers seem to be expected to understand how to interpret these provenance blobs, to know how to assess provenance and permit or not permit accesses to files.  This is an untenable requirement, unless the NFS spec also discloses how to interpret the provenance blobs.

Otherwise, you're implying that there's a new requirement that blocks the implementation of an NFS server that allows for local access to exported file system data.

		Craig

On 9/28/18, 12:21 PM, "nfsv4 on behalf of Chuck Lever" <[email-masked] on behalf of [email-masked]> wrote:

---

From: Chuck Lever <[email-masked]>
Date: Fri, 28 Sep 2018 11:28:43 -0700

> On Sep 28, 2018, at 9:41 AM, Everhart, Craig <[email-masked]> wrote:
> 

The policy mechanism that assesses whether file content is safe
to access is applied by a separate security module, not by the
NFS stack. The security module determines the format of the file
provenance information and sets a policy for deciding when
file content is safe to use locally. The integrity check is
end-to-end and includes NFS but also other parts of the storage
chain.

I am not suggesting that NFS do anything more than transport this
information and prevent its modification by unprivileged users.
This is in fact how NFS already works for file content, named
streams, and extended attributes.

As in other integrity-related proposals in our industry, the
storage platform is not in a position to impose a particular
hash or cryptographic algorithm on the upper layers to measure
integrity. We can and should leave that as an area for others
to innovate, as we have done with GSS for example.

This document is a work in progress, not something that is ready
to advance to publication. If you have specific suggestions,
there is plenty of time to address them. Otherwise I recommend
that you browse IMA-WP and wait and see how my draft improves in
the next few months.

--
Chuck Lever

---

From: "Everhart, Craig" <[email-masked]>
Date: Fri, 28 Sep 2018 18:45:55 +0000

Hi Chuck,

You give the good example of how an NFS server interacts with local processes that observe and manipulate the file content.  It's true that NFS standards don't dictate how such local processes are to interpret file attributes such as their names and their content, but you're suggesting that there are rules that must be followed about how files are to be handled--whether content is safe for manipulation; that requires interpreting the provenance blobs.  We might have to go to standards like POSIX to understand how file permission bits are to be interpreted, but at least there is such a standard.

In this case, you're positing the existence of a security module without explaining how one interacts with it, besides being its faithful channel.  I can't know how to interact with it, or learn from your document where to go for details, yet those interactions are apparently some of the semantic requirements of the extension.

		Craig

    

    

    
    --
    Chuck Lever

---

From: David Noveck <[email-masked]>
Date: Fri, 28 Sep 2018 20:58:42 -0400

> information.

I now understand that, but that doesn't mean that people are not waiting
for it.  Without  a definition of the format somwhere, implementations
cannot be written and if implementations cannot be written, there is no
point in doing this.

> This information is _opaque_ to NFS,

As I understand it, it is opaque to the server, but for there to a be
client implementations
there needs to be code written to create the provenance information and
verify it and
that ccode cannot be written without the format being known.

> and at least in IMA's case,

As I understand things the chances of there being any other case is quite
low, and I
believe that that is your feeling as well.

> there is no published normative specification that defines it.

Perhaps not, but one would hope that there is definition of the format
somewhere.   If there isn't, how
could implementations be developed and interoperate?

> There is no practical way to specify the format in this document.

What makes it impractical?

> reviewers have been confused on this point.

Confused or disbelieving.

> What more can the draft do to make it clear?

The normal function of a specfication like this is to explain how
interoprating implementations can be developed.  If, as I understand to be
the case, you feel that this document or some successor draft cannot do
that, I think you need to clearly state that it will not do that and
explain how you expect interoperating implementations to be developed.

On Fri, Sep 28, 2018 at 12:20 PM Chuck Lever <[email-masked]> wrote:

> > Within NFSv4, file provenance information is treated as opaque data
> which need not be interpreted by the server.  In order for multiple clients
> to co-operate in generating and verifying file provenance information,
> there needs to be a definition of the format in which the information is to
> be stored, preferably in a self-describing format.  This document does not
> resources required for the operations defined here, the length of the file
> provenance information is strictly limited.  It MUST  NOT exceed 4096 bytes
> in length.
> > With regard to the second paragraph, I'll just note here that the phrase
> "credential that is unauthorized to replace the FATTR4_FILE_PROVENANCE
> attribute" appears, leaving the implementer in  doubt about when he "MUST
> return NFS4ERR_ACCESS".   I'll make suggestions on dealing witth this issue
> when we review 5.  Security Considerations.
> since file provenance information is not a name.  I believe NFS4ERR_INVAL
> is a better fit.
> > The fourth paragraph as written does not deal properly with the case in
> which some filesystems on the server support file provenance information
> and other do not.   It indicates that NFS4ERR_type is to be returned in
> this case, when NFS4ERR_ATTRNOTSUPP should be returned.  Suggest the
> following replacement:
> supports FATTR4_FILE_PROVENANCE but object does not support this attribute,
> the server MUST respond with NFS4ERR_WRONGTYPE.  In the case in which the
> file system does not support this  attribute, the error NFS4ERR_ATTRNOTSUPP
> is returned.
> paraphaph of 4.2.  Storing File Provenance Metadata.  Suggest the
> following replacement:
> supports FATTR4_FILE_PROVENANCE but object does not support this attribute,
> the server MUST respond with NFS4ERR_WRONGTYPE.  In the case in which the
> file system does not support this  attribute, the attribute bit is not set
> in the response and no error is returned.
> point of saying the server "MUST NOT" do something, that he has no possible
> way to do.   He doesn't understand the format of the provenance data so it
> basically impossible for him to do a provenance verification.  Saying he
> MUST NOT prevent access based on verification results is confusing since
> there is no point in saying you MUST NOT do something you cannot do.
> information.   As a result, if provenance veification is to be done, it
> must be done by the client, who needs acces to the file data and provenance
> information provided by the server, who is charged with providing each of
> these to the client independently.
>
> > The first paragraph states a REQUIREMENT on the server with no way for
> the server implementor to figure out what he is supposed to do to conform
> to the REQUIREMENT.   I don't see the point of doing this.
>
> > Since the actual security requirements are not known now,  it is best to
> indicate that and wait until the requirements are known to put in the
> RFC2119 language once they are.  Alternatively, you could put in some
> preliminary requirements that could be used for prototyping and indicate
> that they are subject to change.
>

---

From: Chuck Lever <[email-masked]>
Date: Sun, 30 Sep 2018 12:21:12 -0400

> On Sep 28, 2018, at 8:58 PM, David Noveck <[email-masked]> wrote:
> 
> As I understand it, it is opaque to the server, but for there to a be client implementations
> there needs to be code written to create the provenance information and verify it and

It is opaque to the NFS client implementation as well. There is
a security module on each system where FPI is used that reads
the FPI and determines whether to prevent access to the file's
content. It is entirely independent of the file system in which
the FPI and file content is stored.

> As I understand things the chances of there being any other case is quite low, and I

> Perhaps not, but one would hope that there is definition of the format somewhere.   If there isn't, how

IMA has no specification. It has a web site and a white paper and
an implementation in Linux. I cite the web site as an Informative
reference and use IMA as an example. As far as I can tell, that's
the best that can be done.

The IMA web site and white paper state that IMA uses an HMAC.
HMAC is defined by RFC 2104, but that's an Informative document.
I cite that RFC as an Informative reference as well.

Thus I have no normative specification of the format that IMA
uses that I can use in a normative specification of an extension
to NFS.

But aside from that, the FPI format is application-specified, and
the file system does not care what that format looks like, as I
state in the new Introduction below. No file system cares what FPI
looks like.

> The normal function of a specfication like this is to explain how interoprating implementations can be developed.  If, as I understand to be the case, you feel that this document or some successor draft cannot do that, I think you need to clearly state that it will not do that and explain how you expect interoperating implementations to be developed.

I don't disagree with this, but I stated well before posting this
revision that I planned to remove discussion of IMA from this
document and state that the FPI format is defined elsewhere and is
self-describing. There was no objection at that time.

Now suddenly neither of those is adequate. But OK: I take the
review comments to mean I've removed too much context. I've
rewritten the Introduction to include the details about the
layering that were removed along with the IMA-specific language.
I hope this is a step toward addressing the stated concerns.

1.  Introduction

   The security of software distribution systems is complex and
   challenging, especially as software distribution has become
   increasingly decentralized.  An end administrator needs to trust that
   she is running executables just as they are supplied by a software
   vendor; in other words, that they have not been modified by malicious
   actors, contracted system administration services, or broken hardware
   or software.  Software vendors want a guarantee that customer-
   installed executables that fall under support contracts have
   similarly not been modified.

   There already exist mechanisms that protect file data during certain
   portions of a file's life cycle:

   o  Whole file system checksumming can verify so-called Golden Master
      installation media before it is used to install the software it
      contains.

   o  File or block integrity mechanisms can protect data at rest on
      storage servers.

   o  For a distributed file system such as NFS, transport layer
      security or a GSS integrity service (as described in [RFC7861])
      can protect data while it traverses a network between a storage
      server and a client.

   A more extensive mechanism is needed to guarantee that no
   modification of a particular file has occurred since it was created,
   even perhaps after several generations of copies have been made of
   the file's content.

   This guarantee can be accomplished by separately preserving a keyed
   hash, such as an HMAC [RFC2104], of a file's content.  The checksum
   and its signature are verified as the file's content is read into
   memory immediately before it is used.  If verification fails, access
   to the file's content is prevented.  The hash is updated and re-
   signed only when the file is legitimately modified.

1.1.  Architecture Basics

   A keyed hash authenticates the identity of the last modifier of a
   file's content and serves as a strong check of the content's
   integrity.  For the purposes of this document, we refer to this
   metadata using the generic term "file provenance information".

   File provenance information is generated and signed by a "provenance
   authority", and then placed in the file system using special tools.

   A security module separate from the file system stack specifies the
   format of the file provenance information and enforces a policy for
   utilizing it to determine when a protected file's content is safe to
   use on the local system.  For the purposes of this document, we refer
   to this module as a "provenance assessor", and the policy it uses as
   the "provenance assessment policy".

   NFS acts as a conduit by which file provenance information and file
   content move between storage on an NFS server and the provenance
   assessor where that content is to be accessed.  NFS peers accessing a
   set of shared files must all agree on the at-rest file provenance
   information format.  The format is specified by the provenance
   assessor and is therefore not described in this document.

   The protocol extension in this document enables the storage and use
   of file provenance information to protect files stored on an NFS
   server.  The Linux Integrity Measurement Architecture (IMA) is an
   example of a mechanism that provides a full provenance assessment
   service [IMA-WP].

   A Trusted Platform Module [TPM-SUM] can seal the key material used to
   sign and verify file content.  Distributing and protecting such key
   material is outside the scope of the OPTIONAL extension specified in
   this document.

--
Chuck Lever

---

From: Benjamin Kaduk <[email-masked]>
Date: Sun, 30 Sep 2018 22:28:16 -0500

> the FPI and file content is stored.

"Agreement on the interpretation of the contents of this opaque blob is out
of scope for this specification" is always hard to feel good about writing,
though the IETF has done it in the past and will do it again.

> I cite that RFC as an Informative reference as well.

FWIW, RFC 2104 is listed on https://datatracker.ietf.org/doc/downref/ so
there would be no issue with having a normative dependency from a
standards-track document.  (Not that this really helps for the case of IMA,
of course.)

-Ben