Subject: change of persistent_fh attribute (attempt 2)

From: Spencer Shepler <[email-masked]>
Date: Thu, 04 Nov 1999 17:48:53 -0600

Sorry for the extra email but trying to get things turned around.

Here is the next version of the persistent_fh replacement based
on comments from Mike.

----

Change 'persistent_fh' attribute to:

fh_expire_type	2	uint32	READ	Server uses this to specify
					filehandle expiration behavior
					to the client.

const	FH4_PERSISTENT		= 0x00000000;
const	FH4_NOEXPIRE_WITH_OPEN	= 0x00000001;
const	FH4_VOLATILE_ANY	= 0x00000002;
const	FH4_VOL_MIGRATION	= 0x00000004;
const	FH4_VOL_RENAME		= 0x00000008;

FH4_PERSISTENT - fh is valid until object is removed; will not EXPIRE

FH4_NOEXPIRE_WITH_OPEN - fh will not expire while client has file
  open.  If this bit is set, then the other volatile related bits do not
  impact expiration while the file is open.  Once the file is closed or
  if the FH4_NOEXPIRE_WITH_OPEN bit is false, the rest of the volatile
  related bits apply.

FH4_VOLATILE_ANY - fh may expire at any time.

FH4_VOL_MIGRATION - fh may expire during file system migration.  May
only be set if FH4_VOLATILE_ANY is not set.

FH4_VOL_RENAME - fh may expire due to a rename.  This includes a
rename by the requesting client or a rename by another client.  May
only be set if FH4_VOLATILE_ANY is not set.

----

Servers that use volatile filehandles should deny a RENAME or REMOVE
that would effect an OPEN file or any of the components leading to the
OPEN file.  In addition, the server should deny all RENAME or REMOVE
requests during the grace period upon restart.

For FH4_NOEXPIRE_WITH_OPEN, there needs to be a longer explanation
about the requirements behind being able to provide this type of
guarantee.  As Neil pointed out, this requires that the server either
enforce the no RENAME/REMOVE rule or keep mappings available at the
server.  Since this guarantee includes server restart, the OPEN file
information must be saved in stable storage as well.

---

From: Neil Brown <[email-masked]>
Date: Fri, 5 Nov 1999 11:27:49 +1100 (EST)

> only be set if FH4_VOLATILE_ANY is not set.

Could there be some explanation about how the client could make use of
the last two.  I cannot seem them being, from the clients
perspective, functionally different from FH4_VOLATILE_ANY (as rename
and migration could happen at any time).

BTW, I am quite happy with dropping the GETATTR('filehandle') idea. I
saw it primarily as a performance thing, and if needed can be added
with a minor version.

On that note, Mike, your minor versioning proposal says:

	         Note that this rule doesn't mean we can't add bits to
	flag fields, such as new attributes to GETATTR's bitmap4 data

This seems to imply that any bit field, such as persistent_fh and the
various bit fields to do with ACLs, may be extended by a minor
version.  Is this true? Should it be explicit?

NeilBrown

---

From: Mike Eisler <[email-masked]>
Date: Thu, 4 Nov 1999 17:22:40 -0800 (PST)

> and migration could happen at any time).

We should call these the Linux bits. :-)

The notion behind FH4_VOL_RENAME is that I can cache file handles as long as I
don't rename my files. if I do rename my files then I should tack a
LOOKUP/GETFH onto the end of RENAME compound. Of course, another client could
rename files from under me, if I'm concered about that, then I'll treat
FH4_VOL_RENAME as FH4_VOLATILE_ANY, i.e. cache path names in my fh caches.

If FH4_VOL_MIGRATION on, the client will cache the path name of every file is
caches an fh for, but if FH4_VOL_RENAME is off it need not append a
LOOKUP/GETFH to the RENAME. If the client doesn't support migration, then it
need not do anything special at all.

FH4_VOLATILE_ANY simply makes it very clear how tenuous things are. I'd use
FH4_VOLATILE_ANY when doing file transfer to a DOS floppy, but unless
FH4_NOEXPIRE_WITH_OPEN, is also set I'd think twice about writable file access.

> version.  Is this true? Should it be explicit?

I'll make it explicit. Any other feedback?

	-mre

---

From: Neil Brown <[email-masked]>
Date: Fri, 5 Nov 1999 16:35:49 +1100 (EST)

> LOOKUP/GETFH to the RENAME. If the client doesn't support migration, then it
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> need not do anything special at all.

Is a compliant client allowed not to support migration?

The closest the draft seems to come is in section 6 where it says:

   The client will use this information to redirect its requests to the
   new server.

I appreciate that this is "will" rather than "SHALL", but the intent
seems to be that a compliant client will support migration.

I think I have the distinction worked out.  Does this fit your
understanding?

1/  No application has a file open:
      In this case I am doing, or have just done, a LOOKUP when I got
      an error.  I think NFS4ERR_FHEXPIRED is much the same as
      NFS4ERR_STALE - my name cache is out of date, re-try the lookup
      from higher up.

2/ An application has a file open:
      My only possibly way to survive NFS4ERR_FHEXPIRED is to do a
      LOOKUP of where I last saw the file.  The different values of
      volatile_fh tell me what my chances of success are:

        FH4_VOLATILE_ANY : No promises.
        FH4_VOL_RENAME : Unless I just did a rename, almost certainly
		        no chance of success, but even if I did, no
			promises.
        FH4_VOL_MIGRATION: well... given that the reason for the
                  FHEXPIRED wasn't a rename, it will probably work,
		  but no promises.

They all are variations on "No promises".  I would atleast like
another value:
	FH4_VOL_LOOKUP_OK: Guarantee: If the name still exists then it
			   	IS the same file.
This could be used on read-only filesystems, and any filesystem, such
as the server-pseudo-filesystem, which does not support DELETE or RENAME.

[[Note that in the above a current-working-directory needs to be
  considered as the application having a file open]]

If, as a client, the best I could get was "no promises" then I
probably wouldn't want to 'cd' to such a filehandle, wouldn't want to
write to such a file, and the only way I would read from one is to
cache the whole file before returning success/failure to the
application for its open() call (which is pretty much how a browser
might work).

> I'll make it explicit. Any other feedback?
The only other thing I can think of is on much the same lines...
If adding bits doesn't "modify the structure", maybe extending an
enumeration doesn't either?
So, for example, I could add another open_claim_type4 which allowed me
to open a file given a filehandle for the file, rather than directory
handle plus file name.
This seems to be allowed under the "binary compatibility" banner, but
would make life awkward for sniffers - which is also mentioned as a
bit of an issue.

Other than that, it expresses pretty much what I assumed was the
intent.

NeilBrown

---

From: Mike Eisler <[email-masked]>
Date: Fri, 5 Nov 1999 07:04:55 -0800 (PST)

> seems to be that a compliant client will support migration.

If the client is a Java applet, and is told to redirect, that will generate a
security exception.

> 		        no chance of success, but even if I did, no

If FH4_VOL_RENAME was the only FH4_VOL bit on, then you'd never get NFS4ERR_FHEXPIRED, unless another client renamed the file.

>                   FHEXPIRED wasn't a rename, it will probably work,

The promise exists as long as there isn't a migration, and as long as
FH4_VOL_RENAME is off. If the file set is my home directory, I can live
with that, because I expected my IT service organization to tell me
in advance when they move my home directory.

> 	FH4_VOL_LOOKUP_OK: Guarantee: If the name still exists then it

Such an absolute guarantee can made if the file set is on a readonly medium
like CD-ROMs. As such, it might not get a lot of use, even on servers that
exported a CDROM, because there might not be a way for server's file system
APIs to indicate that situation.

In practice, we've found that client-side failover between servers that share
file systems read-only -- but not necessarily mounts them read-only -- works
pretty well.

Having an attribute that indicates if the client has read-only access to the
entire file set seems generally more useful, and if we had such an attribute, I
wonder if that might adequately serve the function of FH4_VOL_LOOKUP_OK.

The below refers to the minor versioning.

> would make life awkward for sniffers - which is also mentioned as a

I see no reason to preclude extending (or reducing, provided the deleted
values aren't used as discriminators) an enumeration, and will make that
explicit.

	-mre

---

From: Neil Brown <[email-masked]>
Date: Thu, 11 Nov 1999 09:45:26 +1100 (EST)

> If the client is a Java applet, and is told to redirect, that will generate a

Ok.

  "The client should, if possible, use this information to redirect its requests to
  the new server."

The "if possible" is probably un-necessary.  And it may be policy, not
possibility, that precludes the use of the information.
> in advance when they move my home directory.

Ahhhh - the light dawns...(I think)  You are talking about risk
assessment, and I was only interested in no-risk scenarios.
How about:

  "The client may use this information [FH4_VOL*] to decide if the
  risk involved in working with the file in question is deemed to be
  an acceptable risk."

and maybe

  "The client-agent may choose to take precautionary actions to reduce
  the risk, such as caching the whole file on the first read, or
  denying write access to it's applications.  If supported, such
  behaviour should be configurable."

As an side, being a member of an "IT service organization", the thing
that I liked about volume migration was that I would be able to do it
*without* having to tell my customers.  Ofcourse, in that case, I
would have to provide a migration mechanism that provided either
FH4_PERSISTENT or  FH4_NOEXPIRE_WITH_OPEN.

> wonder if that might adequately serve the function of FH4_VOL_LOOKUP_OK.

On reflection, I was thinking particularly of virtual filesystems,
where the file name *IS* the key to the file - rename/link not
supported and don't make sense.  As there aren't very many of these
around, it is hard to argue convincingly, so I won't push the point,
but one, hopefully not too far fetched, scenario that I came up with
involves OPENATTR.
OPENATTR presents a mini-filesystem containing the named attributes of
a file.  It is not unreasonable to imagine that some implementations
of named attributes will not support rename or link.
Quite possibly, a named attribute implementation  would store
attributes in such a way that the name was the ONLY reliable
key. (e.g. packed name/length/content list).  The only way to return a
persistant file handle for an attribute file would then be to store
the name in the filehandle, and that may not fit in general.

So we assume that the server returns volatile file handles that would
not normally survive restart.  The server needs to provide the client
with a guarantee that if the server restarts in the middle of a
transaction, then the client can get back to the same file.  The
underlying filesystem supports this (there is a reliable key to the
file) but the only way that the server can currently provide a
guarantee to the client is with FH4_NOEXPIRE_WITH_OPEN, which requires
it to store extra state to stable storage.  With FH4_VOL_LOOKUP_OK, it
could tell the client "If you do the same lookup from the
(persistant) filehandle that you got from OPENATTR, then you can
safely use that to continue the transaction".

As this only impedes efficiency, and does not stop the server from
providing correctness, I am happy to wait and see in any
implementations surface that could actually benefit from LOOKUP_OK.

NeilBrown