Subject: RE: Size of stateid's

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 28 Oct 1999 08:09:27 -0700

> On Wednesday October 27, [email-masked] wrote:
> > In our discussion at the bake-off, I think we were
> > leaning to the view that 64 bits was not big enough
> > for a stateid, or at least that it seems kind of
> > tight.
> > 
> ( This reply is also in response to the earlier article by Dave about
> stateids).
> 
> 
> Is it just my imagination, or are stateids starting to look a lot like
> file handles?

Certainly there is a similarity.

> 
> 1/ Stateids are allocated by the server and used by the client, just
>    like file handles.
> 2/ Stateids need to be bigish, and probably contain a structure that
>    is opaque to the client, just like filehandles.

The above at least justifies referring to statehandles to stress
the similarity.
 
> 3/ Stateids must accompany every IO request, just like file handles.
> 4/ Stateids might get DEADSTATE (STALE?) or BADSTATE (FHEXPIRED?)
>    responses, much like file handles.
> 5/ A stateid uniquely identifies a file (plus some other info) just
>    like a filehandle.

As the protocol now stands, 5/ is not necessarily true.  I think
you could use the same bits for two logically different stateid's
associated with two different files.  This is because currently
the server has the filehandle to disambiguate any overloading.

The implementation that I proposed as an example does have the 
property that you want.  My feeling is that making this
property (that stateid's map uniquely to files) a requirement
would not unduly inconvenience servers, but it would be a change.

If we don't increase the statetid size from 64 bits, it may be 
that some servers would have to use the same stateid values to
mean different things on diefferent files.  Instead of the 
index field being a unique id, it would be a value that
identified the specific lockowner and that this together with
the filehandle would be hashed to get a table index.

I think adding 5/ as a requirement would not be onerous
in the context of variable-length statehandles. 

> 
> Obviously there are differences too, I think that everywhere a
> filehandle is used, as stateid would contain enough information to be
> used instead.

They could contain enough information.

> 
> So the conclusion that this observation seems to bring me to is:
> 
>  There are two types of "filehandles" (though actually is another
>  dimension to the typing of filehandles. We already have some sort of
>  persistant/volatile distinciton).  We will call them "open" and 
>  "non-open" file handles.

Informational RFC on the taxonomy of filehandles?  :-)

It seems that open filehandles would always be volatile.  They
would always become invalid on reboot even if the underlying
non-open filehandle is persistent.

This means if this was done then I would have to implement
a form of volatile filehandle, which I've sworn never to do.
Gag.

> 
>  LOOKUP (and LOOKUPP and PUTROOTFH) will return a "non-open" 
> file handle.

Wait a minute.  There was some text in the spec about using a 
filehandle obtained from LOOKUP to do IO, as opposed to one
obtained from OPEN.  You (at least I think it was you) pointed
out that this doesn't make any sense, that it is the presense
or absence of a stateid which is the issue.

That text would make sense in the conetxt of something like your
proposal.  So, either your proposal was actually in effect
and dropped for some reason (anybody rememeber?) or ...

Somebody with access to this mailing-list has been traveling 
back in time, hacking into Spencer's system and doing some
midnight editing.  I think we have material for an X-files 
episode. :-)

> 
>  All "state changing" requests (OPEN, LOCK, LOCKU, CLOSE, DELEGRETURN)
>  don't return a stateid, but instead set the current filehandle.  The
>  "input" file handle must be "open" except for OPEN. The "output"
>  filehandle will be "open" except for "close". This output filehandle
>  is understood to refer to the same file as the "input" filehandle,
>  but with extra (or less) locking state. 
>  (CLOSE doesn't really need an output file handle, but is pleasingly
>  symetric is it provides one)

I'm not sure about CLOSE.  The fact that close returns a stateid
has been bothering me.  Is this because a single lockowner might
do more than one OPEN (possibly with different state reservations)
or is there some other reason?  I guess this orthogonal to whether
stateid and filehandle are merged.

> 
>  read/write requests no longer need a stateid.  The relevant locking
>  state is already encoded in the filehandle.
>  This would also serve to allow a 
>     open/lock/read
>  sequence in one compound op, which previously was not 
> possible because
>  there was no way to get the stateid from the open into the lock and
>  then into the read.
>  A read/write request on a "non-open" filehandle would be treated like
>  a read/write with an all-zeros stateid (or maybe an all-ones
>  stateid).

Are you also saying that you can use an open filehandle for a 
non-IO request and that this is equivalent to using the underlying
non-open filehandle?  I assume so.

In this case, open filehandle would look a lot like volatile 
filehandle, except that you wouldn't recover using the name.
When you got DEADSTATE, you would go back to the underlying
non-open filehandle, reclaim your locks and use a new open
filehandle.  Even if you couldn't get a new open filehandle,
you could use the underlying non-open filehandle for any
request that doesn't require an open filehandle.

> 
>  You would probably still need that same range of error responses
>  (STATE, FHEXPIRED, GRACE, DEADSTATE, BADSTATE, OLDSTATE, EXPIRED,
>  NOLOCK) to indicate in what way the filehandle was not accepted.
>  It might be nice for some of these to be returned by "PUTFH" rather
>  than the actual operation.

We'd have to be careful in distinguishing those that had to be
checked at PUTH, those that could be checked at PUTFH or the actual
operation, or those that had to be checked at the actual operation.

Examples:

    DEADSTATE shoud be returned at PUTFH.  A handle should never 
    become dead within a COMPOUND.

    OLDSTATE might be checked at PUTFH but given non-atomicity,
    most implementations would also have to deal with the 
    possibility that a lock request got in and made the open
    handle invalid.

    NOLOCK couldn't be checked at PUTFH time since it depends
    on what you are doing.

I think the current reporting of filehandle errors BADFH, STALE,
etc. needs to be made more consistent in any case.  I think thete
are similar issues.

> An obvious implementation would be for the server to simply catenate
> the non-open filehandle and the stateid, and for the client to treat
> the "open" filehandle much like it currently treats the "stateid",
> except when actually building an RPC request.

Another obvious implementation would be to make open-filehandles
look exactly like the stateid's I proposed.  The index would
bring the server to a table which would have the corresponding
non-open-filehandle or a pointer to it.  Actually sending the
persistent filehandle would be redundant.

> 
> Given this, you don't have to decide in advance how big a stateid
> should be.  The server chooses a size.

If we are going to merge stateids with handles, then variable-length
makes sense.  If we aren't then I'd want a fixed size (12 or 16 bytes).
Making something variable-length adds four bytes over-the-wire.
If some people need sixteen, making it variable-length would
require everybody who needed twelve to use sixteen while 
requiring those who need sixteen to use twenty.
  
> 
> You might want the extend the maximum size of a filehandle some
> more, but with the current max of 128 bytes, the client will
> probably be dynamically allocating space anyway.
 
Or at least dynamically allocating for the ones that are larger
than some typical value.

This is interesting idea.  I don't see a Fatal Flaw, at least
yet.