Subject: RE: changes to nfs4_prot.x / protocol description

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 5 Oct 2001 13:55:35 -0700

That's my understanding.

-----Original Message-----
From: Dan Trufasiu [mailto:[email-masked]]
Sent: Friday, October 05, 2001 4:32 PM
To: 'Noveck, Dave'; 'Kendrick M. Smith'; [email-masked]
Subject: RE: changes to nfs4_prot.x / protocol description

The stateid used to be an opaque structure for the client. Do I have to
consider that the new stateid4 is a read only structure (for the client)?
>From my understanding: if I return back a stateid I have to return back what
I received from the server and I don't have to update the  seqid field. I'm
correct?

Thanks

Dan

> -----Original Message-----
> From: Noveck, Dave [mailto:[email-masked]]
> Sent: Friday, October 05, 2001 3:11 PM
> To: 'Kendrick M. Smith'; [email-masked]
> Subject: RE: changes to nfs4_prot.x / protocol description
>
>
> I think I was the one who originally proposed this
> and it was to deal with a specific problem in one of
> the proposals that Spencer made to deal the fork
> problem.  I had been assuming that this issue still
> applied to Spencer's current approach, but I haven't
> verified that.
>
> The issue applied if the scope within which open
> stateids were resolved (i.e. the universe of current
> OPEN such that for a given file all such open merged
> and thus for which a single CLOSE would be appropriate),
> was larger that the scope of requests subject to a
> single seqid (the old-fashioned seid) sequence.  In
> such an environment, two OPEN's for the same file
> or that after the fact turn out to be for the same
> file (don't you just love hard links?) could be
> performed in parallel and with purely opaque stateids
> there would be no way for the client to determine which
> one was generated later and thus has priority.
>
> I haven't yet tried to put the new nfs_prot.x together with
> Spencer's last mail message on this revision of the
> locking/open model yet, so that I can really say that I
> understand how all this will work, but that's the background,
> as far as I understand it.
>
> -----Original Message-----
> From: Kendrick M. Smith [mailto:[email-masked]]
> Sent: Friday, October 05, 2001 2:51 PM
> To: [email-masked]
> Subject: RE: changes to nfs4_prot.x / protocol description
>
>
>
> According to the update, stateid4's are now defined as:
>
> struct stateid4 {
> 	uint32_t	seqid;
> 	opaque		other[12];
> };
>
> Assumption: 'seqid' does not refer to the seqid associated with the
> nfs4_lockowner, but refers to the (formerly opaque) portion of the
> stateid which is mutated on open/close/lock.
>
> If this is correct, why was it decided to expose this part of the
> stateid to the client?  I would actually prefer to have stateid's
> be completely opaque (typedef opaque stateid4[16]) for two reasons:
>
> 1.  Delegation stateid's do not mutate, and therefore have no
> associated seqid field.
> 2.  It seems more natural, given that stateids 0 and -1 have special
> semantics (as pointed out by Dan from Hummingbird).
>
> Is there some reason that I'm missing for making the "stateid seqid"
> visible to the client?
>
> Kendrick
>

---

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 6 Oct 2001 06:09:29 -0700

The URL for Spencer's email is:

http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2001/0428.html

-----Original Message-----
From: Spencer Shepler [mailto:[email-masked]]
Sent: Friday, October 05, 2001 5:05 PM
To: [email-masked]
Subject: Re: changes to nfs4_prot.x / protocol description

On Fri, Sergey Klyushin wrote:

> Where is description of new OPEN/LOCK model? I can not find it on
> http://www.nfsv4.org/rfc3010updates/index.html.
> Looking email list I found only one Spencer's email about new model.
> Is there more detailed specification?

My email of a awhile back is the only text description of the ideas
behind the new model and the updated .x at this point.  I will work
over the weekend to come up with a reasonably short but full
description of how a client and server should use the new data structs
and interact with one another.  We can then use that (in expanded
form) for updates to RFC3010 proper.

-- 

- Spencer -

---

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 10 Oct 2001 11:25:49 -0700

I'll take a stab at #1.

I think you use the open_stateid if the issuing process has
not done any LOCK requests for that file.  If it has done
any, you use the last lock_stateid returned by LOCK or 
LOCKU.

I'm not thrilled with this answer.  The issue that I see 
with it is that it allows an undetected  violation of 
Robinson's stateid ordering mechanism.

In the RFC3010 protocol, if I use a stateid in a WRITE (the
READ case is similar but fairly harmless), and the WRITE
hangs around in the network long enough so that a set of
LOCK and UNLOCK requests occur on that file so that the
locking environment is now different from when the request
was issued, the WRITE request"s stateid will show this and it
will be rejected with OLD_STATEID.

In the October protocol, this is mostly true as well, except
in the case in which the WRITE is issued before *any* LOCK
requests.  In that case, when the out-of-date WRITE appears
it will have the open_stateid, rather than an out-of-date 
lock_stateid and I'm assuming that that can't be rejected
with OLD_STATEID.

To avoid this, I suppose you could require that you use the
lock_stateid if any process has gotten a lock_stateid based
on that open_stateid, which might requrie some processes
to do dummy LOCK/UNLOCK requests to get a lock_stateid when
they might not otherwise have one.

So I guess I'll raise the following:

1'. What stateid is used in READ and WRITE requests when your
    process has done no LOCK/UNLOCK requests but other processes
    have done so based on the same open_stateid.  Is it valid to
    use an open_stateid or must this process obtain a lock_stateid?

1''. What stateid is used in READ and WRITE requests when your
     process has a lock_stateid but has done an open-stateid-changing
     request more recently (e.g. OPEN_DOWNGRADE, OPEN on an existing
     file returning updated open_stateid).

-----Original Message-----
From: Sergey Klyushin [mailto:[email-masked]]
Sent: Tuesday, October 09, 2001 1:53 PM
To: [email-masked]; [email-masked]
Subject: RE: changes to nfs4_prot.x / protocol description

1. What stateid is used in READ and WRITE requests? open_stateid or
lock_stateid?
2. Do we need verifier in OPENres? Why stateid is not enough?

Sergey

---

From: "Kendrick M. Smith" <[email-masked]>
Date: Mon, 15 Oct 2001 22:21:13 -0400 (EDT)

> > > > (2) Consider this scenario with three lockowners L1, L2, L3.
> > > > Initially L1 has a file open with (open) stateid A, and L2 has
> > > > the same file open with (open) stateid B.  The new LOCK operation
> > > > is used to derive lock stateid C, for lockowner L3, from open
> > > > stateid A.  The client then attempts to use the LOCK operation
> > > > again to derive a new lock stateid, for lockowner L3, from open
> > > > stateid B.  Does this:
> > > >   (a) succeed, and instantiate new lock stateid D, which will
> > > >       be independent from lock stateid C,
> > > >   (b) succeed, but not instantiate a new lock stateid (the
> > > >       server would return an updated version of stateid C), or
> > > >   (c) fail with an error?

At the risk of beating this issue to death, I found an example which
I believe shows that (c) won't allow correct implementation of Posix
clients, and that (a) is the only correct choice.

Consider this scenario:
  1. Parent process P opens a file on file descriptor fd1, then forks
     child process C (which inherits fd1).
  2. Child C places a lock on fd1.
  3. Child C closes fd1 (however, P still has it open).
  4. Child C reopens the file on file descriptor fd2.
  5. Child C places a lock on fd2.
  6. With C still holding the lock, parent P closes fd1.

Let's see how this scenario results in disaster if (c) is chosen above:
  1. Lockowner P would transmit an OPEN, instantiating open stateid S1.
  2. Lockowner C would transmit a LOCK, instantiating lock stateid L1
     (derived from S1).
  3. The close() here is a no-op as far as NFSv4 is concerned; rather
     than sending a CLOSE to the server, the client will just decrement
     an internal refcount (since P still has the file open).  (Of course,
     the client would still need to send a LOCKU to release the lock
     acquired in 2.)
  4. Lockowner C would transmit an OPEN, instantiating open stateid S2.
  5. Assuming that (c) is chosen above, the client does not have the
     option to instantiate a new lock stateid derived from S2 (this would
     result in an error).  Rather, it must use the old lock stateid L1
     in the LOCK request.
  6. This close() destroys the last reference to the file table entry,
     so it is not a no-op; rather it results in an nfsv4 CLOSE of stateid
     S1.  This in turn destroys lock stateid L1 and the lock held by C.
     So C has unexpectedly lost the lock on the file!

On the other hand, if (a) is chosen, then in step 5 the client can derive
a new lock stateid L2 from S2 and avoid the "disaster" in step 6.  I have
convinced myself that the new state model will allow correct
implementation of the Linux client, _provided_ that choice (a) is made in
the original example.  So I'm now heavily pushing choice (a) as Spencer
originally suggested.  Comments?

- Kendrick