Subject: RE: Some issues with stateid's

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 29 Oct 1999 08:30:39 -0700

> On Thursday October 28, [email-masked] wrote:
> >  
> > > Does this imply that the funciton of the sequenceid can 
> be subsumed by
> > > the stateid in its current form?
> > > The sequenceid is there precisely to guarantee "at-most-once"
> > > semantics.  It does this by defining an ordering that 
> both the client
> > > and the server must acknowledge to each other. (server returns
> > > "next-in-order" number, client acknowledges it by using 
> it in the next
> > > request).   The stateid provides a similar ordering, and 
> so guarantees
> > > at-most-once semntics.
> >  
> > Certainly not as things stand right now. 

I just thought of one other issue for this approach.  It is
basically an extension of the issue I brought up a few days
ago regarding spece management for per-lockowner structures.
As things currently stand, per-lockowner structures need to
hold a sequence id to compare to the sequence id in any
forthcoming requests for that lockowner.  The problem is
that the server has no way of knowing when a given lockowner
is gone and dispensing with the data structure since it
must keep the sequenceid to compare with a possible subsequent
request for that lockowner.

If stateid take over the function of sequencid's than the 
problem migrates and instead of having lockwoner structures
which you have to keep around, you have the problem for 
each lockowner-file pair.  This is much worse.  A given 
lockowner can easily open millions of files (not all at the
same time usually).  If stateid's stay as they are, then
on the CLOSE, we can return some distinguished dummy stateid
and get rid of the storage associated with that lockowner-
fiel pair.  If stateid's take on the sequence function, we 
can't.  You have to keep the stateid, so it can determine
whether a subsequent OPEN of that same file is correct.

It's true that we don't have a solution to lockowner space
management problem anyway, but solving it for lockowner-file
pairs seems a lot harder.
 
> > 
> > The biggest issue is OPEN.  OPEN now takes a seqid and a lockowner
> > but not a stateid.  You are presumed not to have one with OPEN 
> > being the way you get the first one.  This is why the seqid is
> > per-lowckowner, while stateid's are sequenced
> > per-lockowner-per-file.
> 
> Yes, I see.  Hadn't thought of that side of it.
> But this begs the question:
>   "Can a given lockowner have a given file open twice?"
> 
> If "no", then we could tell open to take a stateid, and give the first
>    open of any file a stateid of 0.  Subsequent opens get the stateid
>    returned from the previous close (or if the server rejects it, a
>    stateid of 0... might need a bit of work)

If you can't have multiple simultaneously active OPENS for 
the same lockowner-file pair, then I think the idea of having
a special initial stateid (like zero) will work.

> 
> If "yes", then...
>    Would this mean that there might be two active stateids for one
>    lockowner/file pair?

I think the answer is no, stateids are per lockowner/file pair.

>    If "yes" then we expose the "interesting" Posix semantic that locks
>       are shared among all file descriptors for the one process.  I
>       would rather not expose one API so blatently in the protocol.

I don't think having multiple stateid's for a given lockowner/file
pair is a good idea.  That's not the way the protocol is now and
I don;t see the benefit.

I don;t follow how this would expose any semantics however.  The
semantics are up to the client.  The server assumes all locks for
a give nfslockowner are shared.  It is up to the client to determine
what constitutes an nfslockowner and set the lockowner value when
making requests.

>    If "no", then we can require the open to take a stateid which must
>       be the "current" stateid.
> 
> However, if "no", (that is, a lockowner can only have a file open
> once), then:
>    How does a client implement:
>      fd1 = open("/a/file", O_RDONLY);
>      fd2 = open("/a/file", O_RDWR);
> 
>    and still get the ACCESS_READ/ACCESS_WRITE/ACCESS_BOTH state right?
>    It really needs to be able to "reopen", or "reshare" or something,
>    which sounds a lot like calling open again, but with a 
> non-zero stateid.

Or like calling open again with no stateid and the appropriate
sequenceid, as would be the way to this as the protocol stands.
 
> It seem to me that there are two choices:
>   1/ Allow a lockowner to have two active stateids on the one file,
>      and expose the details of posix locks in the protocol, (and
>      thereby break WIN32 Locks? I don't know, but I guess so).
> 
>   2/ Require the Client(*) to do any intra-lockowner negotiation and
>      appear to open each file just once (per lockowner), with the
>      option of changing the "access" type without closing the file.
> 
> I like option 2 even if we keep sequenceids, and I think it means that
> we don't need them.

I don't see why these are the only options.  What about explicitly
allowing multiple OPEN's of a file for a given lockowner just
as we allow multiple LOCK's for a given file-lockowner pair.

> 
> (*) I mentioned once before that the term "client" as used in the spec
> seemed awkward, as people have other understands of the word "client".
> What works for me, is to think of an nfs "client" as an "agent" which
> supports one or more "applications".
> 
>  A client might be a Unix system with an Agent in the kernel and
>  various applications running in processes and communicating via a
>  POSIX interface.
> 
>  OR it mighht be a web browser with the Agent in a library and the
>  only application being the rest of the browser.
> 
> Then, because people already have an idea of what an "NFS client" is,
> that is slightly different from our usage, we just don't use that word
> much, and stick with "NFS agent" which is the bit that we are
> interested in.
> 
> Does that work for anyone else (and does it really matter?-)

Doesn't matter to me.

> 
> > 
> > There are some other issues.  The sequencing of stateid's is
> > opaque to the client while seqid's are known to the client.
> > This would allow a client to send N and N+1 without having to
> > wait for the response to N, if it felt the transport was such
> > that it preserved message order with sufficient probability to
> > make this desirable, while with stateid's you couldn't issue
> > N+1 until you you got back the response to N, since the client
> > doesn't know what the next stateid is.  I suspect that this 
> > might be a problem for some clients.
> 
> Given that lock requests require "consecutive" stateids as well as
> sequential sequenceids, a client (agent) couldn't batch multiple lock
> requests like this for the one file.  You could only batch multiple
> lock requests for different files, or maybe multiple open requests for
> the same file (but see above).
> 
> The only time this might make sense is if the client wanted to lock
> parts of a few different files, and wanted the locks to be claimed
> strictly in order, but didn't want to suffer a round-trip latency for
> each lock.  Sounds like a job for a compound op?

Never mind, I was just a bit confused.  Given that you are 
splitting up the sequencing from a per-lockoner to a per-lockowner-
file-pair basis, the fact that the client can't issue-ahead because 
that stateid's are opaque has no negative consequences.  

> 
> (by "consecutive" stateids I of course mean ids of consecutive states,
> and not anything about the numerical properties of stateids.)