Subject: COMPOUND and the nfs reply cache

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 30 Sep 1999 18:00:28 -0700

Working on COMPOUND for the bake-off, I've run into some interesting issues
that I wanted to bring to everybody's attention.  I'm not sure that they affect
the spec too much if at all, but I wanted to warn anybody who is tempted to
use COMPOUND too creatively.  The server, at least my server, might not
be able to cope and return NFS4ERR_RESOURCE.

In the good old days, requests such as READ (and also READLINK and
READDIR) did not cache their results to be returned in the case of 
retransmitted requests, although they might be dealt with in the reply
cache to avoid executing a retransmitted request while the original is
in the process of being retransmitted.  One doesn't need to cache the
results because no fs state changes (everybody, by common consent,
ignoring atime issues) so that if a retransmission gets different data
from the orignal request, the new data is just as right as the original.
Caching the result for these procs would be extremently onerous
because a very large amount of data would have to be saved so
people don't do it.

Now, back to the future.  With COMPOUND, you can have a READ
op, combined with ops that change fs state.  There are two cases 
to consider.  The state-modiying op may be either before or
after the READ.  If it is before, you have a chance.  You can avoid
saving the results.  You do have to save the results of the state-modifying
op, and the current file handle for the READ.  This does add some
complexity since you now have to deal with retransmission in
which you partially replay and partially re-execute the request.  In the 
bake-off time frame, if you do a READ, READLINK or READDIR that
follows a state-modifying op, I'm going to return NFS4ERR_RESOURCE.

The really bad case is if the state-modifying op appears after the READ.
Then you have to save the results returned by the READ.  A retransmission
that returned the contents of the file at the time of the retransmission
would be wrong.  This is seen most clearly if the later op is a WRITE
into the same file, but I think the problem for any modification of
state since. the READ has to appear to have happened before the
state-modifying op and if that is non-idempotent, we are stuck with
it appearing to have occurred when it was first executed.  I am
going to return NFS4ERR_RESOURCE on any state-modifying op
if I have seen a READ, READLINK, READDIR previously in the
same request.  Right now, I don't see ever doing anything 
different.

Maybe the spec should somehow stigmatize this particular use
of compound as seriously anti-social behavior for which 
NFS4ERR_RESOURCE is an appropriate sanction.