Subject: [nfsv4] Should RELEASE_LOCKOWNER operation be synchronous

From: Dave Chiluk <[email-masked]>
Date: Thu, 21 Mar 2013 10:14:52 -0500

Running this program on a linux client *(ubuntu precise, but upstream
kernels are similar) with an AIX server blocks for 15 seconds.

-----------------------------------------------
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/file.h>
#include <unistd.h>

#define FILENAME "testfile"

int main()
{
	int	fd = open( FILENAME, O_RDWR );

	if( fd==0 )
	{
		fputs( "Failed to open `" FILENAME "'.", stderr );
		return 1;
	}

	printf( "flock() returned %d. Now calling close() ...\n",
		flock( fd, LOCK_EX|LOCK_NB ) );
	close( fd );
	return 0;
}
-----------------------------------------------

I have attached a capture of 3 runs of the above open-close program.
The problem is that the order of the packet operations end up being as
follows.

Client			Server
OPEN
			OPEN-ack
LOCK
			LOCK-ack
LOCKU
			LOCKU-ack
RELEASE_LOCKOWNER
CLOSE
			CLOSE - Error with NFS4ERR_DELAY
			RELEASE_LOCKOWNER-ack
 ... wait 15 seconds due to the NFS4ERR_DELAY
CLOSE
			CLOSE-ack

When AIX processes the RELEASE_LOCKOWNER before the close, no error is
returned.

The problem as I see it is that AIX refuses to close a file until the
RELEASE_LOCKOWNER has been processed.  As I read the RFC this is
incorrect, and AIX should process the close regardless of the state of
the lockowner.

So the question becomes, should the Linux client make the
RELEASE_LOCKOWNER call synchronous?  Or should the AIX server process
the Close regardless of state of LOCKOWNER.

Thank you,
Dave Chiluk

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 21 Mar 2013 15:41:20 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Dave Chiluk
> Sent: Thursday, March 21, 2013 11:15 AM
> To: [email-masked]
> Subject: [nfsv4] Should RELEASE_LOCKOWNER operation be synchronous
> 
>  ... wait 15 seconds due to the NFS4ERR_DELAY CLOSE
> 			CLOSE-ack
> 
> 
> incorrect, and AIX should process the close regardless of the state of the
> lockowner.
> 

There is nothing in RFC3530 that states that use of RELEASE_LOCKOWNER is mandatory before CLOSE so I see no reason whatsoever to change the client behaviour.

Cheers
  Trond

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 21 Mar 2013 15:51:15 +0000

> Of Myklebust, Trond
> Sent: Thursday, March 21, 2013 11:41 AM
> To: Dave Chiluk; [email-masked]
> Subject: Re: [nfsv4] Should RELEASE_LOCKOWNER operation be synchronous
> 
> There is nothing in RFC3530 that states that use of RELEASE_LOCKOWNER is
> mandatory before CLOSE so I see no reason whatsoever to change the client
> behaviour.

Let me rephrase that...

There is nothing in the spec that states that RELEASE_LOCKOWNER has any ordering requirements w.r.t. CLOSE. The only ordering requirements are w.r.t LOCK and LOCKU (release_lockowner is required to return NFS4ERR_LOCKS_HELD).
In fact, NFS4ERR_DELAY isn't even mentioned as a valid return for RELEASE_LOCKOWNER. Our client will handle it, but it isn't optimised for it, nor do I see a reason why we should optimise for it.

Cheers
  Trond

---

From: Dave Chiluk <[email-masked]>
Date: Thu, 21 Mar 2013 11:15:26 -0500

>> mandatory before CLOSE so I see no reason whatsoever to change the client
That's how I read it as well.
> In fact, NFS4ERR_DELAY isn't even mentioned as a valid return for RELEASE_LOCKOWNER. Our client will handle it, but it isn't optimised for it, nor do I see a reason why we should optimise for it.

The Server is responding NFS4ERR_DELAY in response to the CLOSE not in
response to RELEASE_LOCKOWNER.  It's doing so because the AIX server
thinks that a RELEASE_LOCKOWNER needs to be processed before a close can
succeed.  Which as you verified is an incorrect reading of the RFC.

Thanks again.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 21 Mar 2013 16:58:30 +0000

> Sent: Thursday, March 21, 2013 12:15 PM
> To: Myklebust, Trond
> Cc: [email-masked]
> > There is nothing in the spec that states that RELEASE_LOCKOWNER has any
> ordering requirements w.r.t. CLOSE. The only ordering requirements are
> w.r.t LOCK and LOCKU (release_lockowner is required to return
> NFS4ERR_LOCKS_HELD).
> > In fact, NFS4ERR_DELAY isn't even mentioned as a valid return for
> RELEASE_LOCKOWNER. Our client will handle it, but it isn't optimised for it,
> nor do I see a reason why we should optimise for it.
> >
> > Cheers
> >   Trond
> 

Right. I misspoke...

> thinks that a RELEASE_LOCKOWNER needs to be processed before a close
> can succeed.  Which as you verified is an incorrect reading of the RFC.

It depends.

If they are actually requiring RELEASE_LOCKOWNER before a CLOSE, then that's incorrect w.r.t. the spec.

If they are just ordering the RELEASE_LOCKOWNER w.r.t. the CLOSE, then that would be a server implementation quirk that they can indeed enforce using NFS4ERR_DELAY, but the penalty for doing so is going to be poor client performance. As I said, such an ordering isn't required by the spec, so it isn't one that clients need to optimise for.

Cheers
  Trond

---

From: Dave Chiluk <[email-masked]>
Date: Thu, 21 Mar 2013 12:11:48 -0500

> If they are actually requiring RELEASE_LOCKOWNER before a CLOSE, then that's incorrect w.r.t. the spec.

Right.

> If they are just ordering the RELEASE_LOCKOWNER w.r.t. the CLOSE, then that would be a server 
> implementation quirk that they can indeed enforce using NFS4ERR_DELAY,
but the penalty for doing
> so is going to be poor client performance. As I said, such an ordering isn't required by the spec, 
> so it isn't one that clients need to optimise for.

That's true just so long as they don't wait forever.

from http://www.ietf.org/rfc/rfc3530.txt page 224 RELEASE_LOCKOWNER
implementation

   The client may choose to use this operation to ease the amount of
   server state that is held.  Depending on behavior of applications at
   the client, it may be important for the client to use this operation
   since the server has certain obligations with respect to holding a
   reference to a lock_owner as long as the associated file is open.
   Therefore, if the client knows for certain that the lock_owner will
   no longer be used under the context of the associated open_owner4, it
   should use RELEASE_LOCKOWNER.

If the client chooses to not ever send release_lockowner *(which would
be valid according to this), then the AIX server might never end up
closing the file.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 22 Mar 2013 14:01:01 -0400

> be valid according to this), then the AIX server might never end up

Yeah, as far as I can tell:

	send close
	if we get DELAY, keep sending close till it works
	then release any lockowners there were associated with that open

would be a legal client implementation, and would deadlock against that
server.

?

--b.