Subject: RE: [nfsv4] random locking questions

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 3 Jun 2003 10:41:02 -0700

David Robinson worte: 
> Ronald E Bretschneider wrote:
> > 1. I assume that NFS V4 does not preclude reads and writes without a
> > preceeding open. I.e. a client could issue V4 lookup operation to determine
> > a file handle and then use it with stateid of 0s or 1s to read or write. Is
> > it correct to allow this behavior?  Is this valid only to an advisory
> > locking file?

> Yes, and Maybe. Bottom line is that a server may reject any use
> of all 0's or all 1's stateids for any reason.  

I don't see this in the spec anywhere.  The spec leaves a lot of
room for server semantics, but I don't see any special freedom for
the all 0's or all 1's case.  

> But it is expected
> that doing I/O via a LOOKUP and READ/WRITE without an OPEN will
> generally be allowed.

The spec is ambivalent about this issue.  It says:

   To provide correct share semantics, a client MUST use the OPEN
   operation to obtain the initial filehandle and indicate the desired
   access and what if any access to deny.  Even if the client intends to
   use a stateid of all 0's or all 1's, it must still obtain the
   filehandle for the regular file with the OPEN operation so the
   appropriate share semantics can be applied.

On the other hand, if you are always going to do an OPEN, you have
a stateid (other than all 0's) with which to do the IO so there is no
point in defining a stateid of all 0's.

So my server (and all the others I know about) do support this, simply
because clients do do this.  Whether they should feel guilty about this 
is a matter best left to individual implementor conscience. 

> It is up to the server to determine how it
> interacts with other locking mechanisms and if it will allow
> it in the face of an outstanding lock, be it mandatory or advisory.

Well the spec says,

   If the lock_owner performs a READ or WRITE in a situation in which it
   has established a lock or share reservation on the server (any OPEN
   constitutes a share reservation) the stateid (previously returned by
   the server) must be used to indicate what locks, including both
   record locks and share reservations, are held by the lockowner.  If
   no state is established by the client, either record lock or share
   reservation, a stateid of all bits 0 is used.  Regardless whether a
   stateid of all bits 0, or a stateid returned by the server is used,
   if there is a conflicting share reservation or mandatory record lock
   held on the file, the server MUST refuse to service the READ or WRITE
   operation.

> The design encourages I/O operations that will not corrupt an
> active lock, such as reading a read or write locked file but not
> allowing a write to a file with any locks. But again, it is up to
> the server to define the behavior.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Ronald E Bretschneider <[email-masked]>
Date: Tue, 3 Jun 2003 12:32:54 -0700

On Mon, David Robinson wrote:
> [IBM sure asks a lot of locking questions...]

ron> I don't want to violate any protocol, either by asking too many
ron> questions or by misunderstanding the specification. I hope this
ron> is ok. :)

> of all 0's or all 1's stateids for any reason.  But it is expected
> generally be allowed. It is up to the server to determine how it

ron> given that there is no implicit lock, a stateid of 0s
ron> is simply an admission by the client that it holds no locks.
ron>   The spec says "A lock may not be granted
ron> while a READ or WRITE operation using one
ron> of the special stateids is being performed and the range of the lock
ron> request conflicts with the range of the READ or WRITE operation".
ron> That leads me to believe that implicit locks were at least
ron> at least encouraged, if not mandated. How can a server
ron> insure that a conflicting lock would not be granted without
ron> implicitly obtaining the lock itself?

> >2. mandatory vs advisory locking questions:
> >   1. it seems that the stateids (as an indication of what locks are
held)
> >   are only interesting if V4 reads or writes are issued for a mandatory
> >   locking file. If access is done to an advisory locking file, a byte
> >   range access conflict is determined during the lock operation. V4
read
> >   and write operations to an advisory locking file should not check for
> >   valid lock ranges being held.  If the server did check, there is no
> >   error code (not_locked for example) to indicate the error. Is this
> >   correct?
>
> No, there is value in checking for advisory locks to specifically catch
> the case where a READ/WRITE gets reordered around a LOCK operation. The
> stateid will allow the server to determine if the operation is from
> a client that is participating in locking or not.

Reordered or the READ/WRITE request is retransmitted.  It could also
be the case that the client is doing something like read-ahead (even
though it should refrain from this behavior when file locks are being
used) and the READ requests are received after a LOCK/LOCKU request.
I see this as a requirement of the server (stateid checking)
regardless of the type of locking being provided (mandatory or
advisory).  This checking and provision for correctness (appropriate
ordering) was one of the main reasons why stateids were added to the
protocol.  In the scenario I just described, the error
NFS4ERR_OLD_STATEID is an appropriate return.

ron> Certainly, I understand if a stateid is used and the sequence number
is
ron> out of order, the request is failed.

The following issue is whether the server checks the actual ranges
after verifying that the stateid is current valid.  This is where the
advisory/mandatory issue arises.  Mandatory says the server is
checking ranges/advisory -- no.  Therefore, if the server is checking
ranges, it is providing mandatory file locking and therefore has the
appropriate error to return.

ron> I thought this was exactly my question. Maybe I didn't say it as
clearly,
ron> but I thought your response says what I intended.

> >   2. it seems for mandatory locking files, if a read or write with a
valid
> >   stateid does not have the requested byte range locked (either none of
> >   the range is locked or only part of the range is locked) an implicit
> >   lock for the duration of the operation should be obtained. I come to
> >   this conclusion because there is no "not_locked" error code on read
or
> >   write. Is this correct?
>
> There are no "implicit" locks in NFSv4 from a protocol definition
> perspective.  A server may choose to use its own internal locking
> structures to serialize requests to manage its internal data structures,
> but that is not visible in the protocol.  An implict lock would
> define that an error could be returned that implied a lock was
> being held when one was not explicitly created, that cannot happen.

David is correct.  I would also suggest search the RFC for
NFS4ERR_LOCKED and read the surrounding text.  It covers the various
scenarios and places the onus on the client to obtain locks if
NFS4ERR_LOCKED is returned as a result of a mismatch between WRITE
range and held LOCK range.

ron> perhaps this section is the most relevant here. This is from the spec:
ron> 9.3.3.  Data Caching and Mandatory File Locking
ron>
ron>   Client side data caching needs to respect mandatory file locking
when
ron>   it is in effect.  The presence of mandatory file locking for a given
ron>   file is indicated when the client gets back NFS4ERR_LOCKED from a
ron>   READ or WRITE on a file it has an appropriate share reservation for.
ron>   When mandatory locking is in effect for a file, the client must
check
ron>   for an appropriate file lock for data being read or written.  If a
ron>   lock exists for the range being read or written, the client may
ron>   satisfy the request using the client's validated cache.  If an
ron>   appropriate file lock is not held for the range of the read or
write,
ron>   the read or write request must not be satisfied by the client's
cache
ron>   and the request must be sent to the server for processing.  When a
ron>   read or write request partially overlaps a locked region, the
request
ron>   should be subdivided into multiple pieces with each region (locked
or
ron>   not) treated appropriately.
ron> This seems to indicate to me that the server should return
NFS4ERR_LOCKED
ron> when a client read/write range is either not covered by an appropriate
ron> lock or partially covered by appropriate locks.
ron> Is this correct? I would not have thought so from the definition of
ron> "NFS4ERR_LOCKED        A read or write operation was attempted on a
ron>                         locked file."

Any comments are appreciated, thanks,
Ron

---

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 3 Jun 2003 15:08:35 -0700

> ron> questions or by misunderstanding the specification. I hope this

Misunderstanding the specification definitely does not violate the
the protocol.  It seems to be inherent to the spec.

> ron> implicitly obtaining the lock itself?

It depends on what you mean by "implicit lock".  If by that you mean
a piece of internal state that prevents for a time invalid combinations
of simultaneous operations, then I think you need something like this
in your server, to deal with the issue above.

The problem is that the implicit lock must not be visible as an nfsv4 
lock, because it isn't one.  Consider the case in which you are doing
a WRITE on a range and then doing a LOCK on that same range.  If this
were two locks that conflicted, then one would fail, while I don't
think you would want the LOCK to fail because of the WRITE.  So any
implicit lock associated with the write has to be invisible to the
clients.  Such a lock can delay other operations but is not the 
same as an actual lock.

Similarly, if you had mandatory byte-range locking and you had two 
WRITEs, implicit locks would serialize the writes making them obey
POSIC atomicity, which is OK if you like that sort of thing :-)  However,
rejecting one of the writes in the same way that would occur with two 
conflicting LOCK's would most definitely not be OK.

<Stuff deleted>

valid
or

when
check
write,
cache
request
or
> ron> lock or partially covered by appropriate locks.

I don't see how it indicates that.  What this indicates is that if mandatory
locking is in effect, the client can use the knowledge of its own locks
to determine when it can use cached data.  Thus, if I have a mandatory
lock on a region, I know another client cannot change it, so I can use my 
cached copy of the data.

I assume that for mandatory locking the server gives NFS4ERR_LOCKED iff
there is conflicting lock for any part of the region.  If the whole
region is unlocked, I am OK.  If I have the appropriate lock, I am
OK.  If someone else has a conflicting lock for the whole region or 
a conflicting lock for a part of it, I get an error.

> ron> "NFS4ERR_LOCKED        A read or write operation was attempted on a

I think "or on region with a conflicting lock held" should be appended.

---

From: Ronald E Bretschneider <[email-masked]>
Date: Tue, 3 Jun 2003 16:42:02 -0700

> > >a file handle and then use it with stateid of 0s or 1s to read or
write.
Is

ron> I totally agree with everything you say. It is exactly what we
ron> intended to do. I was very confused for 24 hours with the idea that
ron> there was no "implicit lock", but you are correct, it is all in
ron> how you define terms (or not).

<Stuff deleted>

valid
or

when
check
write,
cache
request
or
> ron> lock or partially covered by appropriate locks.

I don't see how it indicates that.  What this indicates is that if
mandatory
locking is in effect, the client can use the knowledge of its own locks
to determine when it can use cached data.  Thus, if I have a mandatory
lock on a region, I know another client cannot change it, so I can use my
cached copy of the data.

ron> I agree, I was looking for something that would help me based on
ron> Spenser's suggestion that I review the references to NFS4ERR_LOCKED.
ron> I agree this was a stretch.
ron> It seems that we are expecting that client lockers will be well
ron> behaved (i.e. follow the rules). I am asking what to do with the
ron> client that does not follow the rules. The client passes a valid
ron> stateid, has valid share reservations, is accessing a mandatory
ron> locking file, and writes offset 1000 for a length of 1000.
ron> What error does a server set if the client ONLY holds a lock for
ron> offset 1000 length 500? What error is set if the client ONLY holds a
ron> lock for offset 1500 for a length of 500? What error is set if the
ron> client ONLY holds a lock for offset 3000 for a length of 1000?

ron> Certainly.

ron> Do you have any comment on how you are handling V3 access to a
mandatory locking file?