Subject: Re: Mandatory vs. Advisory

From: Stevan Steve Allen <[email-masked]>
Date: Fri, 17 Jan 2003 15:56:35 -0800

David,

I believe your prior post describes a basic environment where applications
decide if locking (control) is needed and there is no checking by the pfs
level or file server.  In this environment, locking is never required by
the pfs/file server to access data (hence nothing to do with data
integrity).  The NFS supported locking model does not provide data
integrity, instead it is a communication vehicle for remote applications to
control each other.  Customers may attempt<?> to impose data integrity by
controlling access to only those applications which provide the necessary
lock requests for the level of integrity needed by the customer solution.

I have Brent's illustrated book (which we seem to have misinterpreted), do
you recommend any other material describing the control model you mention?

The application control model you describe below is not true in the
"locking" model we are familiar with.

"it is likely that you can take an application written for advisory locking
and have it work correctly with mandatory locking"

I suspect we are incorrectly imposed our locking model onto the NLM/v4
protocols.  Our prior experience comes from an required locking model
designed to guaranteed transactional data integrity in a shared
environment.  In our previous environment, locks were verified at the
remote pfs layer which may reject or provide implicit locking for a file
data request.  In such an environment, an application deciding not to lock
may be incompatible with the pfs layer access controls and fail.

Your application centered control model is interesting and I shiver at the
thought of removing the check in the remote pfs layer thereby allowing
possible data corruption due to a poorly coded application not protecting
data.  But hey, I'm open to new ideas.

Thanks,
Stevan C. Allen

                      Winston Churchill:
" A Riddle wrapped in a mystery inside enigma."

---

From: David Robinson <[email-masked]>
Date: Fri, 17 Jan 2003 19:26:20 -0600

> lock requests for the level of integrity needed by the customer solution.

Maybe it will help to describe an example of how NFSv4 cannot provide
data integrity unless all applications are cooperating. Two processes
are working on a file, both have the file open read/write and also
allow other read/writers (otherwise both couldn't have gotten two
read/write access). In sharing the file each process grabs the
necessary write lock before it modifies the data and drops it when
finished to allow the other process to have access. In the mandatory
locking model, the process can either grab the explicit lock which will
guarentee read access without concern for an interposing write, or it
can simply read the data and if there is a conflicting lock the read
will return an error and the process can decide how to proceed (i.e.
error, sleep and retry, whatever). But even in the case of two
cooperating processes, each explicitly grabbing locks before doing an
I/O, which makes the data between them consistent, there is still the
issue of a third process opening the file and causing corruption.
Because the act of unlocking part of a file and allowing another to grab
a lock is not atomic, the third process can get an operation in
(possibly an I/O without a lock being held) that will corrupt the data.

At best, NFSv4 can only use access control to restrict who
can OPEN a file and thus get access to a file.

> you recommend any other material describing the control model you mention?

The NFSv4 locking model is a hybrid of the standard POSIX advisory
locking model and the Windows file locking model. It does not however
incorporate the SYS V mandatory locking model. Reading publications
on those is probably your best bet.

> data.  But hey, I'm open to new ideas.

I don't know the "pfs" model, but I suspect that you may be used to
a locking model where all I/O operations must be performed under
and explicit lock with strong controls on who can access a file.
That was not put into NFSv4 because it would be difficult to
implement under both Unix and Windows. There was language in earlier
drafts that talked about "implicit" locks, but that led to confusion
and wrong conclusions that an I/O operation caused a lock to be
obtained even without an explicit lock request. In the Unix world
if an application can open a file, it can write all over it and
destroy your data (e.g. "cat foo > bar"). Even worse, if you
restrict access to only known applications, a coding error can
cause an app to read or write data without holding a lock which
could corrupt data, its the Unix/POSIX/Windows way.

I have not seen requests for a stronger locking model, but I can
imagine extending the existing semantics to prohibit an I/O
operation unless the stateid in the request corresponds to
a state where the lock_owner has explicitly gotten the necessary
lock. COMPOUND would make this easy for a client to implement
and it wouldn't be hard for a server to check. But it might
break lots of existing applications. You might be able to
create a vendor specific share/export option that enforces such
really strict behavior by the client, but turning it on would
break interoperability. But even such an option would not
be sufficient to handle the recovery semantics needed, once
NFSv4's lease/grace period has expired new I/O's are
allowed, even if the crash caused a "transaction" not
to complete leading to data corruption.

In general, these sorts of stricting locking semantics have
been punted by the file systems designers to the applications
themselves. As an example, Oracle Parallel server maintains
its own distributed lock manager to achieve tight consistency
and has its own dedicated algorithms to handle recovery.

I suspect you are trying to use NFSv4 in a manner it was
simply not designed for....

	-David

---

From: Mike Eisler <[email-masked]>
Date: Sat, 18 Jan 2003 08:07:20 -0800

> can OPEN a file and thus get access to a file. 

I agree, but your example applies to three processes on
the same system each accessing the same file
on the same local filesystem.

> on those is probably your best bet. 

I disagree. I think the System V mandatory locking model is supportable.
When the server performs aconflicting  I/O on a file with the mandatory
lock bit set, it can use the stateid of the request to see if the
requester has the conflicting lock. If the requester has the conflicting
lock, and the lock doesn't need to be upgraded (e.g. read to write,
range extension), then the server can allow the I/O. Otherwise,
it can simply return NFS4ERR_LOCKED, and the client
can then acquire the necessary lock on behalf of its process.

At least this was how RFS worked in System V Release 3.2. Since the
System V Verification Suite (SVVS) passed on System V R3.2 over RFS
(and not on System V R3.0), I conclude that an NFSv4 client
and server implementing System mandatory locking would
pass SVVS. And that's good enough proof that NFSv4 supports
System V mandatory locking.

BTW, this issue was discussed at length at the Ann Arbor bakeoff
last August, and as a result, text was added to the specification
to clarifiy, as well as eliminating the rather useless mandatory
lock flag from the OPEN results. (It was eliminated because
in the System V world, the mand lock bit can come on
any time. Therefore, NFS4ERR_LOCKED is the only
consistent indicator that mandatory locking is on.

See http://www.nfsv4.org/rfc3010updates/issue218/issue218.html

I don't know what Stevan's client's locking model is, but if the
client represents a significant platform in terms of numbers of users
(given the number of different platforms IBM has, and the size of
IBM's business, a real possibility), future minor revisions of NFSv4
ought to consider extending NFSv4 locking accordingly, if it is feasible.