Subject: RE: REMOVE of an OPEN file

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 24 Sep 2001 11:06:48 -0700

I agree that it is not worth the effort to create a
global mechanism for delete-on-last-close when the 
server does not support delete-on-last-close.

David Robinson writes:
> We could add complexity like adding a clientid to
> remove, but this adds a bunch of server state
> and troublesome recovery semantics.

I don't see that this adds much server state.  The
server has to be able to know what clientid each open
it has is for.  And I don't see any troublesome
recovery semantics.  

The question is the goal here.  This solution, without
the client id, doesn't allow an open file to be removed 
out from under the application and that's good.  It
does make the client do a fair amount of work to make 
reasonably sure that the file eventually gets removed.

Adding the client id reduces the work for the client
implementing the rename hack (realistically he will 
periodically retry), but it does mean that sometimes 
a file will be removed out from under an application
with an open file (i.e. V3-like behavior is possible).

For me, the important thing is that the protocol allows 
servers to implement delete-on-last-close without
creating an unacceptable situation for those that
can't.  I think this proposal does that either with 
or without the clientid on REMOVE. 

-----Original Message-----
From: David Robinson [mailto:[email-masked]]
Sent: Monday, September 24, 2001 1:10 PM
To: [email-masked]
Subject: REMOVE of an OPEN file

"Noveck, Dave" wrote:
[Yet another insightful comment]

I am inclined to try and keep the protocol as simple as
possible in these scenerios and accept that one of the
goals of NFSv4 is to NOT provide a distributed consistent
namespace.

The scenerios with a single client operating on an open
file have been discussed and appear to be fairly clear.
The interesting case appears to be when a server will
not remove an open file and a client other than the
one doing the remove has the file open.

If the server has a file that is open by some client,
or even a local open, it will fail the remove regardless
of whether the remove is from a client doing the temp
file trick or some hapless user doing a /bin/rm of the
file. In either case the remove will fail and the server
should respond with an error indicating it failed due to
the file being open and not some generic permissions problem.

In the latter case we can design a complex mechanism to
create a remove on last global close, but I don't think
it is worth the effort. The client may choose to do some
complex tracking, but as with other parts of NFS, sometimes
the server just won't let the client do what it thinks
it should be able to. So the client must issue an
error to the caller and let the chips fall where they
may.

In the former case, the client knows it is trying to
remove an open file and must assume the removed failed
because it had the file open and has no way of
determining if some other client also has it open as
well.  So it does the old rename hack.  When it finally
attempts to do the remove after it closes the file, it
is in exactly the same situation as described above,
a client removing a file it doesn't have open but the
server is failing the request because it thinks it is still
open. Like the previous case the client must just
determine what is acceptable behavior and what to
do with the application (e.g. return an error on close,
silently just continue. or fork a thread to keep trying
to remove the file).

We could add complexity like adding a clientid to
remove, but this adds a bunch of server state
and troublesome recovery semantics. Realistically
only the client knows what it wants and
trying to communicate the multitude of possibilities
to the server through a couple flags doesn't seem
worth the effort. We may have some orphaned files,
but that already happens with today's rename hack.

So in summary, while Dave's scenerio is valid, I don't think
it is any different than other situations where a server
may refuse a remove. From a protocol specification perspective,
if the server cannot remove an open file, it should
return NFS4ERR_NO_CAN_DO_FILE_OPEN and it is up
to the client to determine what it thinks is best.  It
can choose to error to the application, do the
rename hack, keep polling the server with removes,
or reboot, whatever it thinks is best. If the
server does support removal of an open file it
must track the number of opens and provide the
previously described recovery semantics.

	-David