Subject: [nfsv4] Delegation Recall Problem

From: Peter Varga <[email-masked]>
Date: Thu, 17 Aug 2006 18:40:42 -0400

Who is at fault in the following scenario?

- Client has read delegation on File1
- Client issues SETATTR for File1 with stateid of all 0s
- Server recalls the read delegation to File1 from Client1
- Client responds NFS4_OK to CB_RECALL
- Client never sends DELEGRETURN
- Client is waiting for a reply to the SETATTR on File1
- Client continues to send RENEW periodically

Because the client keeps sending RENEW, the server can not revoke the
delegation.
At this point, the server and client are deadlocked.

Comments, please.

Peter

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Paul Mazzurana <[email-masked]>
Date: Thu, 17 Aug 2006 18:23:48 -0500

If you want to lay blame, I would say the client fault (but then
I'm a server guy ;-) 

In section 9.2 of RFC 3530,

At the time the client receives a delegation recall, it may have
substantial state that needs to be flushed to the server.  Therefore,
the server should allow sufficient time for the delegation to be
returned since it may involve numerous RPCs to the server.  If the
server is able to determine that the client is diligently flushing
state to the server as a result of the recall, the server may extend
the usual time allowed for a recall.  However, the time allowed for
recall completion should not be unbounded.

and then it also says: 

A client failure or a network partition can result in failure to
respond to a recall callback.  In this case, the server will revoke
the delegation which in turn will render useless any modified state
still on the client.

I think you could argue that even though the client is still issuing 
renews, 
the client is not "diligently flushing state to the server", so the server 
can
interpret this as a client failure and revoke the delegation. 

In section 9.2.1, there is additional discussion about the server revoking
a delegation from a failed client in conjunction with support of the
CLAIM_DELEGATE_PREV.  In these cases, the RFC indicates that the
server would need to wait at least the lease period of time before it
can revoke the delegation. 

So, to be safe, I think in the example given, the server can revoke the 
delegation
after the lease period of time has passed after the client responded OK 
from
the recall. 

--
Paul D. Mazzurana
[email-masked]
512-838-3529      Tie: 678

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 17 Aug 2006 19:36:36 -0500

> At this point, the server and client are deadlocked.

The server and client are broken.

The client is obviously broken because it does not send the DELEGRETURN.

The server is broken because it is consuming a resource for the SETATTR
request when it should go ahead and respond with a ERR_DELAY.

The server is also broken because it is not revoking the delegation.

The server should revoke the delegation after onc lease period
has passed.  It should also send a CB_PATH_DOWN error to the client
as a response to the RENEW requests.

Spencer

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 17 Aug 2006 22:24:24 -0400

> as a response to the RENEW requests.

No it shouldn't. It should indeed revoke the delegation, but the
CB_PATH_DOWN error is bogus: the problem isn't that the client's
callback path is inaccessible, but rather that it is failing to complete
the delegation recall in a timely fashion.

It would be more correct to just return NFS4ERR_OLD_STATEID the next
time the client tries to use that delegation stateid.

Cheers,
 Trond

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 17 Aug 2006 21:56:50 -0500

> the delegation recall in a timely fashion.

My apologies.  The server should return ERR_ADMIN_REVOKED to the
RENEW instead of ERR_CB_PATH_DOWN.  I am assuming that the client
is not sending anything other than RENEWs.

Spencer.

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 17 Aug 2006 23:41:09 -0400

> is not sending anything other than RENEWs.

Is that really correct? The definition of ERR_ADMIN_REVOKED is
        
        Due to administrator intervention, the lockowner's record locks,
        share reservations, and delegations have been revoked by the
        server.

RENEW doesn't really have a lockowner.

My other gripes with returning that particular error to RENEW are that 

        - It isn't really going to help the client determine what went
        wrong.
        
        - I always thought that the ADMIN_REVOKED error in RENEW meant
        that the entire lease had been revoked by the administrator (and
        so recovery should normally involve calling SETCLIENTID
        again)...

Cheers,
  Trond

---

From: Spencer Shepler <[email-masked]>
Date: Thu, 17 Aug 2006 22:50:59 -0500

> RENEW doesn't really have a lockowner.

Well, given that delegations are associated with the clientid and
not a lock or open owner, I would interpret the description differently.

>         - It isn't really going to help the client determine what went

Well, in the case that we are discussing the client seems a little
confused in that it has responded positively to the CB_RECALL and
is not taking action to return the delegation so at least the
server is responding with something.

>         so recovery should normally involve calling SETCLIENTID

It is meant to indicate revocation at any granularity; single lock,
entire filesystem, or the entire clientid.

Spencer

---

From: Paul Mazzurana <[email-masked]>
Date: Fri, 18 Aug 2006 08:26:21 -0500

Trond Myklebust <[email-masked]> wrote on 08/17/2006 10:41:09 

>         so recovery should normally involve calling SETCLIENTID

I agree with Trond on this, ERR_ADMIN_REVOKED should not be returned
on the RENEW operations, for the error is for the particular state data 
used,
the clientid, which would apply to ALL the state the client has. 

If/when the client used the recalled/revoked delegation stateid, it would 
get 
the ERR_ADMIN_REVOKED on that particular operation.