Subject: Re: NFSv4 Advisory vs. Mandatory locking issues

From: Stevan Steve Allen <[email-masked]>
Date: Wed, 22 Jan 2003 13:11:47 -0700

We are examining the ability to implement various locking models in the
NFSv4 server to support our various customer needs.  I am posting here to
verify we are not disturbing any NFS rules or verbiage.  Please read the
entire post before replying to specifics.

This post is attempting to answer the following two questions.  With the
introduction of NFSv4 mandatory locking support, what are the implications
to unix applications using advisory locking in v3 and v4 environments, and
what are the implications of sharing a file between v3 and v4 users.

This post also introduces an important distinction that an NFSv4 server
customer may now have the ability to decide which locking model is to be
utilized for his solution.

-----------------------------------------

With the introduction of v4 mandatory locking support, nothing has changed
for applications using NFSv3 (locks are still not checked and temporary
implicit locks will not be obtained by the server on the clients behalf).
To support customer solutions where mandatory locking is required, NFS
versions supporting NLM can not be used.  (e.g. A v3 enabled server will
not verify or obtain the implicit lock needed to support the customers
mandatory locking solution)

Comment: I do not find a problem scenario where the solution is to provide
both NFS v3 advisory locking and NFS v4 mandatory locking to simultaneously
access the same file.  Allowing both advisory & mandatory access may be the
default server environment but I see no benefit or problem being solved
with this approach.

Mandatory locking model.  Mandatory locking was introduced in NFSv4 to
support customer solutions which could not be satisfied using an advisory
locking model.  Describing NFSv4 as a hybrid locking model is not
recommended, instead NFS attempts to provide a conduit for existing locking
models to be negotiated between a server and client, where current and
future enhancements to NFS locking are attempts to support most or all
common locking models implemented by various platforms and allow a given
model to be negotiated between dissimilar client/server platforms.

In NFSv3, the need to lock can not be controlled or enforced by the NFS
server.  Unlike v3, v4 now allows the customer to implement on the server
platform the ability to restrict which locking models are allowed in the
distributed environment based on a files access needs, platform
requirements, or the requirements of a given solution.  The NFSv4 protocol
now provides an NFS server customer (probably the storage admin) the choice
to decide on a file by file, file system, or NFS server basis the locking
model needed by a given solution, and enforce that model from the server
end.  This is contrary to the UNIX world where the application decides if
locks are needed.  NFSv4 locking therefore allows the server customer to
implement platform specific locking solutions which may refuse access to
clients using different locking solutions.  This is not true in an NFSv3
environment.

We are examining the following NFSv4 server solution.  Verbiage may be
specific to our platform.

The NFSv4 draft does not describe (or exclude) the differences between
various mandatory locking models implemented on various server platforms.
On a file by file basis, an NFS server customer may select one of the
following locking models supported by NFSv4.

  o Advisory Locking
  o Mandatory where a lock will be obtained on behalf of the client
    for implicit lock operations (using advisory locking)
  o Mandatory where implicit locking is not allowed (fail request)
    option describes a solution where strong locking is needed.

An interesting clarification is that a v4 client/appl does not know
beforehand if the server is supporting advisory or mandatory locks.  An
unprotected request will not conflict with locks if the server is running
v3, or v4 in advisory mode.  If the request is v4 with the server
implementing mandatory w/ implicit locks, the unprotected i/o request may
now fail (conflicting lock) where it would have been successful using
advisory locking.  If the request is v4 with the server implementing
mandatory w/ only explicit locks allowed, an unprotected request will
always fail.  If the server has disabled NFSv2/v3 access and has
implemented a v4 mandatory locking solution, v2/v3 requests will fail while
v4 implicit requests may have the lock obtained on their behalf, or the
request failed if explicit locking is required.

Thanks,
Stevan C. Allen

            Winston Churchill:
"A Riddle wrapped in a mystery inside enigma."

---

From: David Robinson <[email-masked]>
Date: Wed, 22 Jan 2003 14:45:59 -0600 (CST)

I have trouble with the notion of "implicit locking." NFSv4
has no such notion and an I/O operation should not fail
wity NFS4ERR_LOCKED unless there is an explicit lock. Internally
many implementations will acquire a per file mutex to serialize
I/O requests necessary for some file system semantics
such as those specified by POSIX.  However, the
serialization should only cause an increase in latency
for the reply to the client, not an error.

Realistically the NFSv4 server has only two choices:

	1) Implement advisory locking which allows I/O to
	   complete even in the face of a "conflicting" lock
	   
	2) Implement mandatory locking, causing I/O operations
	   to error if an explicit lock conflicts.

There is no third choice of having the server obtain implicit locks.

If you have a client OS that maintains the model that all I/O
operations occur only within the control of a lock, I would
suggest that the solution should be implemented on the
client.  For all I/O requests, form a COMPOUND operation
that obtains the necessary lock, does I/O and unlocks.
For example <PUTFH, LOCK, READ, LOCKU>. While this will not
prevent an uncooperative client from doing I/O without a
lock, through export/share controls you can limit those clients
from having access.  The advantage is that your application will
get the right semantics with a standard server.

With this approach, you could add the non-standard feature to
the server that it will reject any I/O that is not done within
an explicit lock range. This will of course break standard client's
access, but it is much more civilized than trying to implement
"implicit" locks. The latter will result in a standard client
seeing some operations suceed and some fail depending on
what other I/O operations are being performed, even in
the abscence of explicit locks. With this non-standard feature
all I/Os will deterministically fail without an explicit lock.
For non-standard features, this is the lesser of two evils.

	-David

---

From: Stevan Steve Allen <[email-masked]>
Date: Thu, 23 Jan 2003 07:24:59 -0800

Thanks David,  for what I am asking I summarized your post as follows.

For I/O requests in an NFSv4  mandatory locking environment many NFS server
implementations may serialize the I/O requests by acquiring a file mutex.
Mutex describes serialization semantics where each object successfully
completes.  To support mandatory locking solutions, the NFSv4 protocol
introduced locked based failure conditions not supported by the mutex
serialization model for which I/O requests are failed. If an I/O request
passes the locking conditions described in the NFSv4 draft a standard NFS
server implementation will satisfy the requests locking requirements.
Additional lock based failure conditions other than those described in the
NFSv4 draft to support customer solutions such as strong locking are
considered non-standard NFS implementations. Non-standard deviations of the
NFSv4 locking protocol should be documented by the implementing product.
The NFSv4 protocol does not support or restrict the use of non-standard NFS
implementations to satisfy customer solutions.

My view may be narrow in this case, I assume the external behavior for
"implicit locking" is not agreed upon.

Thanks,
Stevan C. Allen

---

From: David Robinson <[email-masked]>
Date: Thu, 23 Jan 2003 11:20:59 -0600 (CST)

> implementations to satisfy customer solutions.

As a clarification, non-standard implementations are by definition
not "NFSv4" and should not be called NFSv4.  This is a common
confusion in the IETF, if you use the term NFSv4 it is
standards conforming, anything else should have it's own
name (e.g. Bob's FS). Otherwise there is no way to distinguish
between an implementation that has changed just one required
feature and another that is loosely based on NFSv4 (think DAFS).
	
> "implicit locking" is not agreed upon.

There isn't any concept of "implicit locking" to be agreed upon. There
is no requirement in the protocol for ordering or preventing
interleaving between I/O requests.  It is just that most
server's implementations have inherent file systems semantics
that provide per request atomicity.

	-David

---

From: Stevan Steve Allen <[email-masked]>
Date: Thu, 23 Jan 2003 12:24:54 -0800

David wrote:

>> The NFSv4 protocol does not support or restrict the use of nonstandard
NFS
>As a clarification, nonstandard implementations are by definition

This is the area I'm unfamiliar with and appreciate your responses.

I support the IETF/WG need to draw the line and define boundaries.  From
the
protocol perspective, the only real "NFS" server product are those that
support all the procedures and provide the expected results outlined in
the draft.  Nothing more, nothing less.

Where we got into trouble is certain platforms may not be able to support a
particular NFS draft item and therefore implemented a failure type
restriction "I can't do that" which is not defined in the protocol.  An
example would be a server not able to support delegation for reasons
entirely
of it's own, or not support file share semantics.

The problem is deciding if this product can still be labeled NFS, or
does IETF/WG not accept this as an NFS product because of the restriction.
The answer of not NFS may affect the goal of NFS acceptance and
availability.
It seems like we are at a point where restrictive offerings may call
themselves NFS without anyone forcing them to change their name to Bob's
FS.
Even though by protocol definition they are not NFS due to their
restrictive
nature.  I'm only describing restricting function, not adding additional
function.

What are the ramifications of incorporating non-NFS restrictions to satisfy
platform limitations or limitations to support customer requirements and
labeling your product as NFS?

The problem
in defining "NFS" products

             -David

---

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 23 Jan 2003 12:57:05 -0800

> David [Robinson] wrote:
NFS
> the draft.  Nothing more, nothing less.

The spec is not very rigid as far as a lot of expected results.  In part,
this is because of the idea that NFS-v4 is not supposed to define 
file system semantics but is merely a protocol so that the client can 
make requests and results as defined by the servers semaantics refelcted
to the client.  I actually think a larger part is due to the fact that 
we'd like the spec to be under a thousand pages.  Nevertheless, the
spec does contain a lot of material that sure seems like semantics to
me.  I'm not saying this apparent contradiction cannot be resolved,
but I am saying that I can only do it in a limited set of cases. 

> restriction "I can't do that" which is not defined in the protocol.  

I think you have to stay within the errors defined for a given op.
Otherwise you are not obeying the protocol.  Usually you can find an
error that sort of works.

> An
> of it's own, 

Delegations are not a required feature.  The protocol is so constructed
that you are not obliged to return delegations.  It is up to the server.
If he doesn't want to return one in any given situation or all situations,
the spec offers no reproaches.

> or not support file share semantics.

I don't see why a server is not able to do this, at least within the
context of other v4 opens?  I can see that it might be problematic to
exclude non-v4 opens but I think the basic idea is that the spec doesn't
cover that (Although angry customers might have their own opinions).

> themselves NFS without anyone forcing them to change their name to Bob's

I've never heard of the IETF forcing someone to change the name of
their product because they returned NOTSUPP is some case not mandated
by the spec.  But I'm not really up on IETF lore.

The problem

             -David