Subject: [nfsv4] LAYOUTCOMMIT (or absence of thereof) semantics

From: Benny Halevy <[email-masked]>
Date: Wed, 24 Feb 2010 16:32:52 -0800

Mike,

The excerpts from rfc5661 below show that the server has the authority
to update the file size using loca_last_write_offset as provided by
the client on LAYOUTCOMMIT or ignore it altogether and use an internal
mechanism such as polling its data servers (having a cluster-coherent
file system would be another example)

========================================================================
12.5.4.2. LAYOUTCOMMIT and size

   The size of a file may be updated when the LAYOUTCOMMIT operation is
   used by the client.  One of the fields in the argument to
   LAYOUTCOMMIT is loca_last_write_offset; this field indicates the
   highest byte offset written but not yet committed with the
   LAYOUTCOMMIT operation.  The data type of loca_last_write_offset is
   newoffset4 and is switched on a boolean value, no_newoffset, that
   indicates if a previous write occurred or not.  If no_newoffset is
   FALSE, an offset is not given.  If the client has a layout with
   LAYOUTIOMODE4_RW iomode on the file, with a byte-range (denoted by
   the values of lo_offset and lo_length) that overlaps
   loca_last_write_offset, then the client MAY set no_newoffset to TRUE
   and provide an offset that will update the file size.  Keep in mind
   that offset is not the same as length, though they are related.  For
   example, a loca_last_write_offset value of zero means that one byte
   was written at offset zero, and so the length of the file is at least
   one byte.

   The metadata server may do one of the following:

   1.  Update the file's size using the last write offset provided by
       the client as either the true file size or as a hint of the file
       size.  If the metadata server has a method available, any new
       value for file size should be sanity-checked.  For example, the
       file must not be truncated if the client presents a last write
       offset less than the file's current size.

   2.  Ignore the client-provided last write offset; the metadata server
       must have sufficient knowledge from other sources to determine
       the file's size.  For example, the metadata server queries the
       storage devices with the control protocol.

18.42. Operation 49: LAYOUTCOMMIT - Commit Writes Made Using a Layout

   The loca_last_write_offset field specifies the offset of the last
   byte written by the client previous to the LAYOUTCOMMIT.  Note that
   this value is never equal to the file's size (at most it is one byte
   less than the file's size) and MUST be less than or equal to
   NFS4_MAXFILEOFF.  Also, loca_last_write_offset MUST overlap the range
   described by loca_offset and loca_length.  The metadata server may
   use this information to determine whether the file's size needs to be
   updated.

========================================================================

In case the client restart case we say:
   In this
   scenario, the data written by the client but not covered by a
   successful LAYOUTCOMMIT is in an undefined state; it may have been
   written or it may now be lost.  This is acceptable behavior and it is
   the client's responsibility to use LAYOUTCOMMIT to achieve the
   desired level of stability.

That said, the spec does not seem to define the server's behavior
for the case the client returns the layout before sending LAYOUTCOMMIT,
either via an explicit LAYOUTRETURN or by returning NFS4ERR_NOMATCHING_LAYOUT
to CB_LAYOUTRECALL, and the case where the client's layout was revoked before
sending LAYOUTCOMMIT.

In either case, I think it's fair to presume that the state of any data
written to the DS, whether it was COMMITted there or not is undefined,
similar to the client restart case, and the spec. should encourage
that server SHOULD use an alternative mechanism, similar to the ones
described in 12.5.4.2. item 2. to recover any data committed to the DSs,
in particular data written and committed to stable storage on the DS.

Benny