Subject: Re: [nfsv4] layout stateid issue

From: Mike Eisler <[email-masked]>
Date: Thu, 21 Feb 2008 17:53:37 -0800 (PST)

diffs are attached.

--- Mike Eisler <[email-masked]> wrote:

> I am going assume we have consensus on this. 
> 
> I also noticed that we do not appear to have any text that explicitly
> says that in order for LAYOUTCOMMIT to be sent, the client MUST
> have a layout. Consensus around the room at the bake-a-thon is that
> it
> is a MUST.
> 
> So when I write the text reflecting the below, I will write text that
> reflects the above.
> 
> --- Robert Gordon <[email-masked]> wrote:
> 
> > 
> > On Feb 21, 2008, at 10:56 AM, Mike Eisler wrote:
> > 
> > > A question came up at the Austin bake-a-thon yesterday.
> > >
> > > Let's say a client uses an OPEN stateid in LAYOUTGET, and gets
> back
> > > a layout stateid. Later, it issues a LAYOUTRETURN which returns
> > > another layout stateid. Let us call this stateid L.
> > >
> > > Sometime later (e.g. days later), the client uses L in LAYOUTGET.
> > >
> > > Question:
> > >
> > > - Between the time L was returned and used, is the server allowed
> > to
> > >   delete L (and thus return NFS4ERR_BAD_STATEID)?
> > >
> > > Or is the server required to keep the stateid around until the
> > client
> > > issues FREE_STATEID?
> > >
> > 
> > I'd return NFS4ERR_BAD_STATEID, iff the client had no outstanding
> > layout segments,  since in 18.43.3 we say
> > 
> > "The loga_stateid field specifies a valid stateid. If a layout is
> not
> >  
> > currently
> > held by the client, the loga_stateid field represents a stateid  
> > reflecting the
> > correspondingly valid open, record lock, or delegation stateid."
> > 
> > presumably the client does not hold a layout since it returned it, 
> 
> > however
> > if it still does have a portion of the file the layout stateid
> should
> >  
> > remain valid...
> > 
> > --
> > Robert
> > "Don't Take Anything Personally"
> > 
> > _______________________________________________
> > nfsv4 mailing list
> > [email-masked]
> > http://www.ietf.org/mailman/listinfo/nfsv4
> > 
> 
> 

Index: nfsv41_middle_op_layoutcommit.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutcommit.xml,v
retrieving revision 1.21
diff -u -w -b -r1.21 nfsv41_middle_op_layoutcommit.xml
--- nfsv41_middle_op_layoutcommit.xml	29 Jan 2008 08:19:32 -0000	1.21
+++ nfsv41_middle_op_layoutcommit.xml	22 Feb 2008 01:48:59 -0000
@@ -17,7 +17,10 @@
       layouts are sub-dividable, a smaller portion of a layout,
       retrieved via LAYOUTGET, may be committed.  The region being
       committed is specified through the byte range (loca_offset and
-      loca_length).
+      loca_length). This region MUST overlap with one or more existing layouts
+      previously granted via LAYOUTGET (<xref target="OP_LAYOUTGET"/>),
+      each with an iomode of LAYOUTIOMODE4_RW.
+      
     </t>
     <t>
       The LAYOUTCOMMIT operation indicates that the client has
@@ -34,7 +37,7 @@
       If the loca_reclaim field is set to TRUE, this indicates that
       the client is attempting to commit changes to a layout after the
       reboot of the metadata server during the metadata server's
-      recovery grace period.  This type of request may be necessary
+      recovery grace period (see <xref target="mds_recovery"/>).  This type of request may be necessary
       when the client has uncommitted writes to provisionally
       allocated regions of a file which were sent to the storage
       devices before the reboot of the metadata server.  In this case
@@ -74,7 +77,9 @@
       last byte written by the client previous to the LAYOUTCOMMIT.
       Note that this value is never equal to the file's size (at most
       it is one byte less than the file's size) and MUST be less than
-      or equal to NFS4_MAXFILEOFF.  The metadata server
+      or equal to NFS4_MAXFILEOFF.  Also, loca_last_write_offset MUST
+      overlap the range described by loca_offset and loca_length.
+      The metadata server
       may use this information to determine whether the file's size
       needs to be updated.  If the metadata server updates the file's
       size as the result of the LAYOUTCOMMIT operation, it must return
Index: nfsv41_middle_op_layoutreturn.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutreturn.xml,v
retrieving revision 1.26
diff -u -w -b -r1.26 nfsv41_middle_op_layoutreturn.xml
--- nfsv41_middle_op_layoutreturn.xml	17 Dec 2007 22:17:39 -0000	1.26
+++ nfsv41_middle_op_layoutreturn.xml	22 Feb 2008 01:48:59 -0000
@@ -131,14 +131,14 @@
     </t>
     <t>
       If the operation specified lr_returntype of LAYOUTRETURN4_FILE,
-      then the lorr_stateid will represent the layout stateid as
+      then lrs_stateid will represent the layout stateid as
       updated for this operation's processing; the current stateid
       will also be updated to match the returned value.  If the 
       last byte of any layout for the current file, client ID, and
-      layout type is being returned and there are not remaining 
+      layout type is being returned and there are no remaining 
       pending CB_LAYOUTRECALL operations for which a LAYOUTRETURN
-      operation must be done as a completing operation, this stateid
-      value may be the special stateid consisting of all zeros. 
+      operation must be done as a completing operation, lrs_present
+      MUST be FALSE, and thus no stateid will be returned.
     </t>
     <t>
       On success, the current filehandle retains its value.
Index: nfsv41_middle_pnfs.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_pnfs.xml,v
retrieving revision 1.73
diff -u -w -b -r1.73 nfsv41_middle_pnfs.xml
--- nfsv41_middle_pnfs.xml	19 Feb 2008 20:09:08 -0000	1.73
+++ nfsv41_middle_pnfs.xml	22 Feb 2008 01:48:59 -0000
@@ -523,7 +523,7 @@
 <t>
   In order to get a layout, the client must first have
   opened the file via the OPEN operation. When a client has
-  no layout on a file, it presents a stateid as returned by
+  no layout on a file, it MUST present a stateid as returned by
   OPEN, a delegation stateid, or a byte-range lock stateid
   in the loga_stateid argument. A successful LAYOUTGET
   result includes a layout stateid. The first successful LAYOUTGET processed
@@ -641,6 +641,15 @@
  before using the seqid. For CB_LAYOUTRECALL arguments, the
  client MUST send a response to the recall before using the seqid.
 </t>
+
+<t>
+
+ Once a client has no more layouts on a file, the layout stateid
+ is no longer valid, and MUST NOT be used. Any attempt to use such
+ a layout stateid will result in NFS4ERR_BAD_STATEID.
+
+</t>
+
 </section>
 
 <section title="Committing a Layout" anchor="committing_layout">
@@ -690,15 +699,21 @@
   between a WRITE to a storage device and the LAYOUTCOMMIT may observe
   a size that does not reflect the actual data written.
 </t>
+
+<t>
+  The client MUST have a layout in order to issue LAYOUTCOMMIT.
+</t>
+
 <section title="LAYOUTCOMMIT and change/time_modify">
 <t>
   The change and time_modify attributes may be updated
   by the server when the LAYOUTCOMMIT operation is processed.  The
   reason for this is that some layout types do not support the update
   of these attributes when the storage devices process I/O operations.
-  The client is capable providing a suggested value to the server for
+  If client has a layout with the LAYOUTIOMODE4_RW iomode on the file,
+  the client MAY provide a suggested value to the server for
   time_modify within the arguments to LAYOUTCOMMIT.
-  Based on layout type, the provided value may or may not be used.
+  Based on the layout type, the provided value may or may not be used.
   The server should sanity check the client provided values
   before they are used.  For example, the server should ensure that
   time does not flow backwards.  The client always has the option to
@@ -733,8 +748,14 @@
   LAYOUTCOMMIT operation.  The data type of lora_last_write_offset is
   newoffset4 and is switched on a boolean value, no_newoffset, that
   indicates if a previous write occurred or not.  If no_newoffset is
-  FALSE, an offset is not given.  A loca_last_write_offset
-  value of zero means that one byte was written at offset zero.
+  FALSE, an offset is not given.  If the client has a layout with
+  LAYOUTIOMODE4_RW iomode on the file, with an lo_offset and lo_length
+  that overlaps loca_last_write_offset, then the client MAY
+  set no_newoffset to TRUE and provide an offset that will
+  update the file size. Keep in mind that offset is not the same
+  as length, though they are related. For example, a loca_last_write_offset
+  value of zero means that one byte was written at offset zero, and so
+  the length of the file is at least one byte.
 </t>
 <t>
   The metadata server may do one of the following:
@@ -1355,9 +1376,8 @@
   protocol.  In general, crash recovery for layouts is similar to
   crash recovery for delegations in the base NFSv4.1 protocol.  However,
   the client's ability to perform I/O without contacting the metadata
-  server subtleties that must be handled correctly if
+  server introduces subtleties that must be handled correctly if
   the possibility of file system corruption is to be avoided.
-  <cref>mre: layouts are bound to stateids</cref>
 </t>
  <section title="Recovery from Client Restart" anchor="pnfs_client_recovery">
  <t>
_______________________________________________
nfsv4 mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/nfsv4

---

From: Benny Halevy <[email-masked]>
Date: Fri, 22 Feb 2008 08:12:59 -0600

>> have a layout. Consensus around the room at the bake-a-thon is that

Mike, I agree with all the modifications you listed.

Benny

> ------------------------------------------------------------------------
>