Subject: RE: [nfsv4] Dealing with issues 16, 17, 25, 34, and 65

From: [email-masked]
Date: Mon, 3 Jul 2006 21:05:11 -0400

If we go ahead with this, some of it (e.g., callback path down)
would appear to be applicable to pNFS.

Thanks,
--David 

> -----Original Message-----
> From: Noveck, Dave [mailto:[email-masked]] 
> Sent: Friday, June 30, 2006 4:38 PM
> To: [email-masked]
> Subject: [nfsv4] Dealing with issues 16, 17, 25, 34, and 65
> 
> This is a proposal to address a number of related issues.  In 
> the issue
> tracker, these are issues 16, 17, 25, 34, and 65, but I don't 
> think that
> the actual protocol issues  necessarily match up very well with the 
> specific items in the issue tracker.
> 
> So it seems to me that the protocol issues can be summarized as
> follows:
> 
>      We have had a number of situations in which we have had a need
>      to notify the client of various sorts of status change.
> 
>      In v4.0, this has resulted in a number of complications 
>      as we hijacked explicit renewal as a way to return these
>      status indications in the form of errors, thereby leading 
>      to restrictions on implicit renewal.
> 
>      There is considerable confusion about these with many 
>      cases in which questions about how things work having to 
>      be answered by very intense reading between the lines of
>      RFC3530.
> 
>      The above leads to anomalies such as errors being returned
>      with the operation performed not being suppressed, as might
>      be expected.
> 
>      Sessions, by carrying the session id on every request, 
>      implicitly identifies the client, thus allowing expanded
>      implicit renewal (on every v4.1 request).
> 
>      The status notifications in the form of errors inherited 
>      from v4.0 interfere with this natural handling of renewal
>      by requiring explicit renewal in many cases.
> 
>      There remains considerable confusion about the errors which
>      are associated with these status indications, including
>      the question of exactly when it is valid to return these
>      errors and whether and in what conditions it is OK to 
>      return NFS4ERR_BAD_STATEID instead ("never", "if you really 
>      have to", "whenever you want" have all been advocated).
> 
>      Related to the above, there is no way to cleanly acknowledge 
>      loss of lock state, which leads to a number of problems.
>      
>      The v4.1 spec continues to refer to the client maintaining
>      clocks and scanning its locks when it think it might have 
>      lost its lease, a kind of second-guessing which would be 
>      unnecessary with a clean method of notifying the client of
>      such conditions.
> 
> So basically my approach to all this is to take advantage of 
> SEQUENCE to return such status indications, get rid of the need 
> for explicit renewal and errors to return such status indications, 
> and then regularize and clarify the handling of locks that have 
> been invalidated by revocation both within and outside the lease 
> period.  Here's an outline:
> 
>      A word of status indications would be added to the response
>      to SEQUENCE.  This would include bits to indicate such things
>      as:
> 
>          Callback path down.
> 
>          Client has had some lock administratively revoked.
> 
>          Client has had a delegation revoked (with no expiration
>          of the lease) because he didn't return it promptly.
> 
>          Client lost one or more sets of locks because of lease
>          expiration without losing all locks (server maintained
>          courtesy lock and then there was a conflict).
> 
>          Client lost all of his locks because of a lease expiration
>          (server not adopting the more restrained policy of keeping
>          locks around).
> 
>       Status indications on SEQUENCE would not be errors and would
>       not stop the COMPOUND and would persist until the condition
>       that gave rise to them changed.  In the case of invalidated
>       locks, this would include deletion of the invalidated lock
>       state via a new OP (see below).
> 
>       The spec would define what the client would do when these
>       status bits were set as follows:
> 
>           If the callback path is down fix it and/or return 
>           outstanding delegations.
> 
>           If client lost all of his locks, either acknowledge 
>           each piece of lock state individually (via 
>           OP_RELEASE_BADLOCK, see below) or all at once by 
>           establishing a new clientid using a new verifier to 
>           start fresh.
> 
>           In the other cases, the client scans all of his locks
>           by stateid, acknowledges the loss if the ones that
>           have been invalidated and continues operation. 
> 
>       The stuff in the spec about the client figuring out if the
>       server might have seen a delay long enough to cause lease
>       expiration would be removed.  If there is lease expiration
>       or loss of locks, the client will find out about them 
>       through the status bits returned by SEQUENCE.
> 
>       Errors used in v4.0 to return such status indications would
>       go away at least for that purpose.  The NFS4ERR_CBPATHDOWN
>       would be invalid.  The errors NFS4ERR_EXPIRED, 
>       NFS4ERR_ADMIN_REVOKED would be retained but only to indicate
>       that the stateid used had been invalidated.  They would not
>       indicate client-wide problems.  Those would be indicated via
>       the status bits in response to SEQUENCE.  A similar error
>       NFS4ERR_DELEG_REVOKED would be added for delegation revocation
>       which is not really the same as administrative revocation.
> 
>       The server would return the appropriate error for invalidated
>       locks based on the reason for invalidation and would not
>       be allowed to return NFS4ERR_BAD_STATEID in this case.
> 
>       A rule would be established that when locks were revoked, all
>       locks covered by a given stateid, are revoked together.  The
>       current spec isn't very clear about this.
> 
>       When there is an indication of lock loss, the client needs to
>       scan his locks for validity.  In order to make this easier,
>       there is a TEST_LOCK op which does nothing but return the 
>       appropriate error if the lock (as represented by the stateid) 
>       has been invalidated. 
> 
>       A new OP, RELEASE_BADLOCK would be used to release an 
>       invalidated lock (as represented by a stateid).  This would
>       not return errors for invalidated locks (of course) but would
>       return an error if applied to stateid without invalidated 
>       locks.
> 
>       One an invalidated locks is released the stateid becomes 
>       invalid and the server is expected to return 
> NFS4ERR_BAD_STATEID.
>       The client is presumed to have obtained the information he
>       needed about the lock by using TEST_LOCK before doing the
>       RELEASE_BADLOCK.
> 
>       Once an invalidated lock is released (and thus acknowledged), 
>       the client MUST NOT attempt to reclaim any such lock.  The
>       server may therefore remove the client from the set of clients
>       (maintained on persistent storage) of clients not allowed
>       to reclaim locks after a server reboot.  Once the client 
>       knows that the lock is gone and will not attempt to reclaim 
>       it, there is no problem with the server accepting reclaims 
>       from that client.
> 
> Questions and comments appreciated.  If there is consensus on this
> general approach, then I will propose detailed changes to the spec
> and send them to a list, so there will still be an opportunity for
> further discussion of the details before we decide that these issues
> are ready to be set to "editing".
> 
>      
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nfsv4
> 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 5 Jul 2006 08:50:42 -0400

With pNFS in particular, it seems quite bad to delay notification
to the next RENEW, expected to be half the lease period away.
Notifying the client on his next MDS request is clearly better
and sessions makes that possible.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, July 03, 2006 9:05 PM
To: Noveck, Dave; [email-masked]
Subject: RE: [nfsv4] Dealing with issues 16, 17, 25, 34, and 65

If we go ahead with this, some of it (e.g., callback path down) would
appear to be applicable to pNFS.

Thanks,
--David 

> This is a proposal to address a number of related issues.  In the 
> issue tracker, these are issues 16, 17, 25, 34, and 65, but I don't 
> think that the actual protocol issues  necessarily match up very well 
> with the specific items in the issue tracker.
> 
> So basically my approach to all this is to take advantage of SEQUENCE 
> to return such status indications, get rid of the need for explicit 
> renewal and errors to return such status indications, and then 
> regularize and clarify the handling of locks that have been 
> invalidated by revocation both within and outside the lease period.  
> Here's an outline:
> 
> general approach, then I will propose detailed changes to the spec and

> send them to a list, so there will still be an opportunity for further

> discussion of the details before we decide that these issues are ready

> to be set to "editing".