Subject: RE: Some issues with stateid's

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 3 Nov 1999 09:12:29 -0800

> On Wednesday October 27, [email-masked] wrote:
> > 
> > All of the stateid fields are OK but one of the locks
>                                        ^^^
> > designated by the stateid has been revoked.  ("Yes,
> > that is our agreement, but I have decided it no longer
> > applies.  So sue me").
> 
> I think the above "one" should be "all".
> Unless there is a reliable way for the client to determine which lock
> (or DENYal, or delegation) was revoked, there seems little point in
> revoking one lock without revoking them all, and I can't see a client
> doing anything useful with the remaining locks when one is gone.
 
Given the stateid architecture, if you've lost one lock you've
effectively lost them all.  Any time you use the stateid you'll
get NFS4ERR_EXPIRED, so you can't use the locks to do IO, you can't
revalidate locks that weren't taken away since using the stateid
will give you an error, you can't release the locks in question.

The question is how the spec should deal with this issue.
For pure revocation, I think we should just define revocation
as happening to the all the locks for a given file-lockowner
pair at the same time, so you would change "one of the
locks" to the "the locks" or "all of the locks".

The other case is lease expiration.  When a lease expires,
the server may let you keep the lock subject to revalidation,
if nobody else wants a conflicting lock.  I have been 
treating losing the lock after expiration, as a kind of
revocation.  In that case, you might lose one of your
locks due to a conflict but, but have others still 
available.  I agree this is of no value so maybe the
spec should say that once you lose one of your locks
due to expiration, the whole set of locks for that
file is revoked.  I think we would need to slightly 
change some definitions and do some rewording.

The most interesting case is if the leases for some of
your locks have expired but others have not.  Suppose
you have a lease interval of 2 minutes and do a lock
at 11:00, another at 11:01, and if there is no IO,
the lease for the lock will expire at 11:02, while the
lease for the second will expire at 11:03 and is subject
to indefintie renewal as long as anyone on the client
does IO.  You could thus have a situation in which
the first lock was given to someone else, at say 11:02:30,
while the second lock still had a valid lease.  I think
the spec has to make clear that the second lock gets 
revoked (even before lease expiration) when the first 
one is given to someone else.  Right now the idea is 
that revocation within the expiration period is a special 
action, due to administrator command, for example, 
so the spec would have to be changed to allow this
to happen as a result of lease expiraion on a another
lock for the same file-lockowner pair.

Another issue is once you get in this state, how does the
server ever get rid of the state it is holding for the
stateid.  Normally, you can get rid of the state
by returning all of your locks (CLOSE+UNLOCK) but since
you can't do that with a bad stateid, it's unclear how 
this stuff could ever be deallocated on the server,
given the protocol as it stands.  This is tied in with
your EXIT/CLOSE suggestions which I'll try to respond to
soon.

---

From: Neil Brown <[email-masked]>
Date: Thu, 4 Nov 1999 13:56:46 +1100 (EST)

On Wednesday November 3, [email-masked] wrote:
> does IO.

This shows you have to read the spec carefully.... I was under the
impression that "any successful use of a stateid" would renew all
locks for all lockowners for the same client. But as you say, it
isn't "any successful use" but is "any successful IO" (Section 8.4).
Given that the I/O is seen as "a positive indication that the client
is still alive", and that a lock request is also an indication that
the client is still alive, should we change:
 
   Any I/O request that has been made with a valid stateid is a positive
   indication that the client is still alive and locks are being
   maintained.  

to something like:

   Any I/O request or Locking or unlocking request with a valid
   stateid, or any open request with a valid clientid, is a positive
   indication that the client is still alive and locks are being
   maintained.

or maybe:

   Any successful use of a valid stateid or a clientid is a positive
   ..... 

That would neatly sidestep the different expiry times, though we still
need a well defined resolution path from lock expiry.  Maybe bad
(NFS4ERR_EXPIRED) stateids should still be allowed for LOCKU
requests.  Afterall, the problem is that the lock is gone, so asking
"please remove this lock" shouldn't be too onerous..

One the topic of lease times though, something else (very small)
occurred to me when I was reading through the debate about lease times
(I think it was "locking errors" september last year).
The issue was that some people thought that lease times should be
long, because they had mission critical locks and frequent network
partitions, and others thought they had to be short for reasonable
recovery times.
It occurred to me that this issue could be addressed by making lease
times a statement about client behaviour rather than server
behaviour.  It is a small change but it may help clarify things.

Rather than saying:
   The server can revoke locks that have not been renewed within the
   lease time - so the client better renew pretty often
we could say:
   The server can do whatever it likes (because we all know that it
   can), but that if the client endevours to renew at least every
   'leasetime', then the server will do it's best-effort to maintain
   the locks for the client.

The effect of this change was highlighted by a comment by Carl Beame 
in 
   http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/0692.html

that client should renew every 1/2 the lease time, just to be sure.
Given that the server can hold on to a lease a little bit longer when
it wants to (such as allowing rebooted clients to reclaim delegations)
we have the situation where the client doesn't believe the leasetime,
and halves it, and the server doesn't believe the leasetime and
extends it.  The change would mean that the client should believe the
leasetime, because it cannot gain by not doing so, and that the server
is expressly permitted to treat the leasetime in a configurable way.

If, we tell the client precisely what to do (renew every leasetime -
more often won't gain you anything), then the server admin can tune
the server behaviour to conditions.
A cautious admin with stable clients, dodgy networks, and plenty of
spare time to chase up stray locks can tune the server to hold on to
clients for hours or days.  A busy admin with stable networks
and no real dependancy on locking could tune it right down so that
locks expire moments after the leasetime.
A default configuration would probably be to hold locks for 2 or 3
times the leasetime.

This could certainly still be done with the current definition of
"leasetime", but that would leave you wandering what exactly it really
meant.

NeilBrown

---

From: Mike Eisler <[email-masked]>
Date: Wed, 3 Nov 1999 19:23:10 -0800 (PST)

>    the locks for the client.

This either semantically the same, or it is semantically vastly different.
if the former, I see no need to churn the specification. If the latter,
I see no need to churn the specification without implementation experience.

Thus far, every objection made to the locking protocol has been successfully
rebutted by David Robinson, and its been going on for about a year.
I think we should leave it alone, until we real flaw is proven by experiment.

	-mre

	-mre