Subject: [nfsv4] Dealing with issues 16, 17, 25, 34, and 65

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 30 Jun 2006 16:38:13 -0400

This is a proposal to address a number of related issues.  In the issue
tracker, these are issues 16, 17, 25, 34, and 65, but I don't think that
the actual protocol issues  necessarily match up very well with the 
specific items in the issue tracker.

So it seems to me that the protocol issues can be summarized as
follows:

     We have had a number of situations in which we have had a need
     to notify the client of various sorts of status change.

     In v4.0, this has resulted in a number of complications 
     as we hijacked explicit renewal as a way to return these
     status indications in the form of errors, thereby leading 
     to restrictions on implicit renewal.

     There is considerable confusion about these with many 
     cases in which questions about how things work having to 
     be answered by very intense reading between the lines of
     RFC3530.

     The above leads to anomalies such as errors being returned
     with the operation performed not being suppressed, as might
     be expected.

     Sessions, by carrying the session id on every request, 
     implicitly identifies the client, thus allowing expanded
     implicit renewal (on every v4.1 request).

     The status notifications in the form of errors inherited 
     from v4.0 interfere with this natural handling of renewal
     by requiring explicit renewal in many cases.

     There remains considerable confusion about the errors which
     are associated with these status indications, including
     the question of exactly when it is valid to return these
     errors and whether and in what conditions it is OK to 
     return NFS4ERR_BAD_STATEID instead ("never", "if you really 
     have to", "whenever you want" have all been advocated).

     Related to the above, there is no way to cleanly acknowledge 
     loss of lock state, which leads to a number of problems.
     
     The v4.1 spec continues to refer to the client maintaining
     clocks and scanning its locks when it think it might have 
     lost its lease, a kind of second-guessing which would be 
     unnecessary with a clean method of notifying the client of
     such conditions.

So basically my approach to all this is to take advantage of 
SEQUENCE to return such status indications, get rid of the need 
for explicit renewal and errors to return such status indications, 
and then regularize and clarify the handling of locks that have 
been invalidated by revocation both within and outside the lease 
period.  Here's an outline:

     A word of status indications would be added to the response
     to SEQUENCE.  This would include bits to indicate such things
     as:

         Callback path down.

         Client has had some lock administratively revoked.

         Client has had a delegation revoked (with no expiration
         of the lease) because he didn't return it promptly.

         Client lost one or more sets of locks because of lease
         expiration without losing all locks (server maintained
         courtesy lock and then there was a conflict).

         Client lost all of his locks because of a lease expiration
         (server not adopting the more restrained policy of keeping
         locks around).

      Status indications on SEQUENCE would not be errors and would
      not stop the COMPOUND and would persist until the condition
      that gave rise to them changed.  In the case of invalidated
      locks, this would include deletion of the invalidated lock
      state via a new OP (see below).

      The spec would define what the client would do when these
      status bits were set as follows:

          If the callback path is down fix it and/or return 
          outstanding delegations.

          If client lost all of his locks, either acknowledge 
          each piece of lock state individually (via 
          OP_RELEASE_BADLOCK, see below) or all at once by 
          establishing a new clientid using a new verifier to 
          start fresh.

          In the other cases, the client scans all of his locks
          by stateid, acknowledges the loss if the ones that
          have been invalidated and continues operation. 

      The stuff in the spec about the client figuring out if the
      server might have seen a delay long enough to cause lease
      expiration would be removed.  If there is lease expiration
      or loss of locks, the client will find out about them 
      through the status bits returned by SEQUENCE.

      Errors used in v4.0 to return such status indications would
      go away at least for that purpose.  The NFS4ERR_CBPATHDOWN
      would be invalid.  The errors NFS4ERR_EXPIRED, 
      NFS4ERR_ADMIN_REVOKED would be retained but only to indicate
      that the stateid used had been invalidated.  They would not
      indicate client-wide problems.  Those would be indicated via
      the status bits in response to SEQUENCE.  A similar error
      NFS4ERR_DELEG_REVOKED would be added for delegation revocation
      which is not really the same as administrative revocation.

      The server would return the appropriate error for invalidated
      locks based on the reason for invalidation and would not
      be allowed to return NFS4ERR_BAD_STATEID in this case.

      A rule would be established that when locks were revoked, all
      locks covered by a given stateid, are revoked together.  The
      current spec isn't very clear about this.

      When there is an indication of lock loss, the client needs to
      scan his locks for validity.  In order to make this easier,
      there is a TEST_LOCK op which does nothing but return the 
      appropriate error if the lock (as represented by the stateid) 
      has been invalidated. 

      A new OP, RELEASE_BADLOCK would be used to release an 
      invalidated lock (as represented by a stateid).  This would
      not return errors for invalidated locks (of course) but would
      return an error if applied to stateid without invalidated 
      locks.

      One an invalidated locks is released the stateid becomes 
      invalid and the server is expected to return NFS4ERR_BAD_STATEID.
      The client is presumed to have obtained the information he
      needed about the lock by using TEST_LOCK before doing the
      RELEASE_BADLOCK.

      Once an invalidated lock is released (and thus acknowledged), 
      the client MUST NOT attempt to reclaim any such lock.  The
      server may therefore remove the client from the set of clients
      (maintained on persistent storage) of clients not allowed
      to reclaim locks after a server reboot.  Once the client 
      knows that the lock is gone and will not attempt to reclaim 
      it, there is no problem with the server accepting reclaims 
      from that client.

Questions and comments appreciated.  If there is consensus on this
general approach, then I will propose detailed changes to the spec
and send them to a list, so there will still be an opportunity for
further discussion of the details before we decide that these issues
are ready to be set to "editing".

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4