Subject: Re: [nfsv4] Validation of state migration

From: [email-masked]
Date: Fri, 3 Feb 2006 16:03:51 -0500 (EST)

> First, the client knows that a file system has migrated - but as Trond points 
> out, how does the client know if the state has also migrated?

I would have thought that the old server can't return NFS4ERR_MOVED or
list the new server in Fs_locations until the migration event is complete.

It would be tempting to use NFS4ERR_DELAY during the migration event, but
NFS4ERR_DELAY is not one of the errors that is exempt from incrementing
the seqid (end of Sec. 8.1.5). I think you're ok, so long as you
return NFS4ERR_DELAY for something that sets the CFH to an fh in the
migrating fs (such as PutFh) and not the Open/Lock/...?

just a guess, rick

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "William A.(Andy) Adamson" <[email-masked]>
Date: Fri, 03 Feb 2006 16:27:52 -0500

> list the new server in Fs_locations until the migration event is complete.

right. the old server waits until the migration event is complete, and then 
reports NFS4ERR_MOVED to the client who then requests the fs_locaions and 
mounts the new server. so the client now knows the fs has migrated (data and 
ACLs at least!) and would like to use the state it established on the old 
server if it could only tell if the migration included the transfer of state 
from the old to the new server.

> migrating fs (such as PutFh) and not the Open/Lock/...?

in a cluster file system situation, where nfsv4 state is all that will need to 
be migrated, the old server can wait until the new server is ready - this can 
be a feature of the YTBD server-server state protocol. i don't see a need for 
NFS4ERR_DELAY.

-->Andy

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 3 Feb 2006 16:44:37 -0500

> list the new server in Fs_locations until the migration event is complete.

I think you're missing the problem: there are two ways (at least) that
rfc3530 gives servers to do migration:

	1) they can just make the client do state recovery on the new server;
	   or

	2) they can migrate all the client's state (so the client can
	   continue using its old stateid's, clientid's, etc. on the new
	   server).

At the time the client gets NFS4ERR_MOVED it doesn't know whether it's
in case 1 or 2.  The only way for it to find out is to just try some of
its old state on the new server (e.g., send a renew with its old
clientid), and see if that works.

If the old state doesn't work, it's in case 1.  But what if it does
work?  That could be because it's in case 2.  Or it could be that the
old clientid worked purely by coincidence, because the new server just
happened to have given out exactly the same clientid to some other
random client.  At which point bizarre awful things could happen.

If the two server implementations cooperate, then they can probably
generate stateid's in such a way that this is guaranteed never to
happen.  Maybe the intention of the rfc is to require that no server
ever return NFS4ERR_MOVED unless it has this level of cooperation with
the new server.  I don't think so, though--I think case 1 was intended
to work between any two NFSv4 implementations without any need for any
special coordination of their clientid's or stateid's.

--b.

---

From: Mike Eisler <[email-masked]>
Date: Fri, 3 Feb 2006 14:59:37 -0800 (PST)

> special coordination of their clientid's or stateid's.

I've always figured that if the old file handles work from server A to the data migrated
to server B, then server A and server B probably won't have issues with stateids and
clientids mistakenly colliding.

If the old file handles don't work, then the state is probably not going to be migrated.
Without an open migration protocol, this seems hard to do without the two servers
agreeing on a stateid formats. If a closed miration protocol is being used, then one would
think the file handle format problem would be worked out too.

So I don't think this is a big deal. That said, deprecating NFS4ERR_MOVED, and replacing
it with two new errors:

  NFS4ERR_MOVED_DATA
  NFS4ERR_MOVED_DATA_AND_STATE

might solve this in NFSv4.1.

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 3 Feb 2006 18:19:48 -0500

On Fri, Feb 03, 2006 at 02:59:37PM -0800, Mike Eisler wrote:
> I've always figured that if the old file handles work from server A to
> the data migrated to server B, then server A and server B probably
> won't have issues with stateids and clientids mistakenly colliding.

So you're suggesting a client could check the FH4_VOL_MIGRATION flag to
determine whether to try using its old stateid's and clientid's on the
new server?

That might have worked.  But since it's not specified that way, it's not
something clients can count on.  On linux at least I think getting
consistent filehandles is easy enough for a cluster filesystem to do
without any cooperation from the nfs server.  It's quite easy for me to
imagine an implementation that could do that but not carve up the
clientid/statied space.

> So I don't think this is a big deal. That said, deprecating
> NFS4ERR_MOVED, and replacing it with two new errors:
> 
>   NFS4ERR_MOVED_DATA

That might help.

--b.

---

From: Mike Eisler <[email-masked]>
Date: Thu, 9 Feb 2006 12:16:48 -0800 (PST)

> > might solve this in NFSv4.1.

I would like to formally propose that NFSv4.1 deprecate NFS4ERR_MOVED and add
NFS4ERR_MOVED_DATA and NFS4ERR_MOVED_DATA_AND_STATE to the list of errors
NFS operations can return (in place of where they are allowed to return
NFS4ERR_MOVED).

Can we wind up discussion by Thursday NOON pacific time, February 16?