Subject: [nfsv4] draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

From: "Noveck, David" <[email-masked]>
Date: Mon, 8 Jul 2013 13:44:03 -0400

This document is getting close enough to expiration (8/19) that we need to consider what will be in the -02.

Originally I had planned to produce a -02 and try to move that forward through the process toward RFC publication.  Given that implementation work has slowed down, I think it makes more sense to delay a bit on that until we have working implementations.

So I will produce a -02 to prevent expiration, but there will probably be a -03 before we try to move the document forward.  If anybody knows of some change to be made or issue to be addressed in the -02, please let me know ASAP and I'll try to make sure it gets in.

I'm not going to present anything about this at IETF87.

---

From: Chuck Lever <[email-masked]>
Date: Mon, 8 Jul 2013 13:47:33 -0400

On Jul 8, 2013, at 1:44 PM, "Noveck, David" <[email-masked]> wrote:

> So I will produce a -02 to prevent expiration, but there will probably be a -03 before we try to move the document forward.  If anybody knows of some change to be made or issue to be addressed in the -02, please let me know ASAP and I’ll try to make sure it gets in.

Have we answered the question of how a server can hold off a chattery client while it is transferring state to the destination server?  Is returning NFS4ERR_DELAY on a file handle-bearing ops sufficient?

> I’m not going to present anything about this at IETF87.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: "Noveck, David" <[email-masked]>
Date: Mon, 8 Jul 2013 15:42:12 -0400

> Is returning NFS4ERR_DELAY on a file handle-bearing ops sufficient?

You can return NFS4ERR_DELAY on PUTFH ops that are within the fs being transferred.

You also have to return NFS4ERR_DELAY on LOOKUP and LOOKUPP ops that cross into the fs being transferred.

RESTOREFH is pain.  I'd prefer it if RESTOREFH could return NFS4ERR_DELAY but servers could work around the issue if it can't.

This will be addressed in the -02.

From: Chuck Lever [mailto:[email-masked]]
Sent: Monday, July 08, 2013 1:48 PM
To: Noveck, David
Cc: [email-masked]; [email-masked]; nfsv4 list ([email-masked])
Subject: Re: draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

On Jul 8, 2013, at 1:44 PM, "Noveck, David" <[email-masked]<mailto:[email-masked]>> wrote:

---

From: Piyush Shivam <[email-masked]>
Date: Mon, 08 Jul 2013 14:48:13 -0500

On 07/ 8/13 12:44 PM, Noveck, David wrote:
>
> This document is getting close enough to expiration (8/19) that we 
> need to consider what will be in the -02.
>
> Originally I had planned to produce a -02 and try to move that forward 
> through the process toward RFC publication.  Given that implementation 
> work has slowed down, I think it makes more sense to delay a bit on 
> that until we have working implementations.
>
> So I will produce a -02 to prevent expiration, but there will probably 
> be a -03 before we try to move the document forward.  If anybody knows 
> of some change to be made or issue to be addressed in the -02, please 
> let me know ASAP and I'll try to make sure it gets in.
>

There is the broader question of returning NFS4ERR_DELAY on NFS  client 
ops that hit the source server while the state transition is still going 
on. There are several ops where NFS4ERR_DELAY  is not a legal error: 
GETFH, OPEN_CONFIRM, RENEW, RESTOREFH, SAVEFH, and RELEASE_LOCKOWNER. I 
can imagine that for OPEN_CONFIRM, RESTOREFH, and SAVEFH, the server 
could return NFS4ERR_DELAY on the preceding file handle bearing op in 
the compound, and the server could even allow GETFH to go through during 
the file system transition, but I was less sure about RENEW and 
RELEASE_LOCKOWNER. RENEW and RELEASE_LOCKOWNER don't have any file 
handle context, and don't have to have any preceding file handle bearing 
op either. One thought is to make NFS4ERR_DELAY a legal error for all 
ops. Would that  be too big of a leap?

Thanks,
Piyush

---

From: "Noveck, David" <[email-masked]>
Date: Mon, 8 Jul 2013 16:07:50 -0400

> Would that  be too big of a leap?

Perhaps not, but I wouldn't try it without a parachute.

From: Piyush Shivam [mailto:[email-masked]]
Sent: Monday, July 08, 2013 3:48 PM
To: Noveck, David
Cc: Chuck Lever ([email-masked]); [email-masked]; nfsv4 list ([email-masked])
Subject: Re: draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

There is the broader question of returning NFS4ERR_DELAY on NFS  client ops that hit the source server while the state transition is still going on. There are several ops where NFS4ERR_DELAY  is not a legal error: GETFH, OPEN_CONFIRM, RENEW, RESTOREFH, SAVEFH, and RELEASE_LOCKOWNER. I can imagine that for OPEN_CONFIRM, RESTOREFH, and SAVEFH, the server could return NFS4ERR_DELAY on the preceding file handle bearing op in the compound, and the server could even allow GETFH to go through during the file system transition, but I was less sure about RENEW and RELEASE_LOCKOWNER. RENEW and RELEASE_LOCKOWNER don't have any file handle context, and don't have to have any preceding file handle bearing op either. One thought is to make NFS4ERR_DELAY a legal error for all ops. Would that  be too big of a leap?

Thanks,
Piyush

---

From: Piyush Shivam <[email-masked]>
Date: Mon, 08 Jul 2013 16:34:18 -0500

Just to be clear, I am not making a case for returning NFS4ERR_DELAY on 
RENEWs, but rather looking for _a_ legal and interoperable way to stall 
the client for _all_ possible ops from the client while the file system 
transition is in play. Clearly, as you pointed out, returning 
NFS4ERR_DELAY for RENEW sure sounds awkward, especially in the general case.

On 07/ 8/13 03:55 PM, Myklebust, Trond wrote:
>
> Returning NFS4ERR_DELAY on RENEW sounds like a recipe for disaster. I 
> can’t see why you’d even want to begin considering to start 
> contemplating it…
>
The larger question in play is: how do we "stall" the client while the 
file system transition is in play. In general the strategy has been to 
return NFS4ERR_DELAY, but that does not work well for ops where 
NFS4ERR_DELAY is not a legal error. What would be your suggestion for an 
error code that a server could return to the client for a RENEW op while 
the file system transition is in play? NFS4_OK is one of the choices 
that the implementors had considered, which I think may work out as well.

> How is the client supposed to know when it needs to retry in order to 
> avoid losing the lease? In most cases, we send RENEW when we’re close 
> to the _/end/_ of the lease period…
>
A client implementation could choose to continue sending RENEWs at its 
usual cadence, because,  RENEW has an inbuilt notion of delay (e.g., 
close to end of the lease). In the special case of file system 
migration, once the state migration completes, a server could hold on to 
the lease till it sees a state renewing op from the client (with certain 
upper bounds of course), at which point the server will stop sending 
NFS4ERR_DELAY to the client for any state renewing op, and the client 
can resume regular business.

That being said, I don't see an advantage of using NFS4ERR_DELAY over 
NFS4_OK (just  yet) in the specific case of RENEW, except it sure would 
be nice to have universal "stalling" op.

> *From:[email-masked] [mailto:[email-masked]] *On 
> Behalf Of *Noveck, David
> *Sent:* Monday, July 08, 2013 4:08 PM
> *To:* Piyush Shivam
> *Cc:* [email-masked]; nfsv4 list ([email-masked])
> *Subject:* Re: [nfsv4] draft-ietf-nfvs4-rfc3530-migration update-01 
> and forthcoming versions.
>
> Perhaps not, but I wouldn’t try it without a parachute.
>
> *From:*Piyush Shivam [mailto:[email-masked]]
> *Sent:* Monday, July 08, 2013 3:48 PM
> *To:* Noveck, David
> *Cc:* Chuck Lever ([email-masked] 
> <mailto:[email-masked]>); [email-masked] 
> <mailto:[email-masked]>; nfsv4 list ([email-masked] 
> <mailto:[email-masked]>)
> *Subject:* Re: draft-ietf-nfvs4-rfc3530-migration update-01 and 
> forthcoming versions.
>
> let me know ASAP and I’ll try to make sure it gets in.
>
>
> There is the broader question of returning NFS4ERR_DELAY on NFS  
> client ops that hit the source server while the state transition is 
> still going on. There are several ops where NFS4ERR_DELAY  is not a 
> legal error: GETFH, OPEN_CONFIRM, RENEW, RESTOREFH, SAVEFH, and 
> RELEASE_LOCKOWNER. I can imagine that for OPEN_CONFIRM, RESTOREFH, and 
> SAVEFH, the server could return NFS4ERR_DELAY on the preceding file 
> handle bearing op in the compound, and the server could even allow 
> GETFH to go through during the file system transition, but I was less 
> sure about RENEW and RELEASE_LOCKOWNER. RENEW and RELEASE_LOCKOWNER 
> don't have any file handle context, and don't have to have any 
> preceding file handle bearing op either. One thought is to make 
> NFS4ERR_DELAY a legal error for all ops. Would that  be too big of a leap?
>
> Thanks,
> Piyush
>

---

From: Piyush Shivam <[email-masked]>
Date: Mon, 08 Jul 2013 18:20:01 -0500

Agreed. Moreover, for the case where there is more than one mount to the 
server from the client, and only one of those mounts is affected by the 
file system migration, returning DELAY is even more confusing. It looks 
like if we return NFS4_OK on RENEW, and the server provides a fresh 
lease on the destination after the file system transition completes, I 
think we can handle the RENEW across a file system transition cleanly.

Barring additional issues, RELEASE_LOCKOWNER seems to be the only 
remaining problematic op during file system transition.

On 07/ 8/13 04:37 PM, Myklebust, Trond wrote:
>
> Right, but my point is that doing so in the particular case of RENEW 
> is bad as it leaves the lease timeout completely unanchored.
>
> *Sent:* Monday, July 08, 2013 5:34 PM
> *To:* nfsv4 list ([email-masked])
> *Cc:* Myklebust, Trond; Noveck, David; [email-masked]
> Just to be clear, I am not making a case for returning NFS4ERR_DELAY 
> on RENEWs, but rather looking for _a_ legal and interoperable way to 
> stall the client for _all_ possible ops from the client while the file 
> system transition is in play. Clearly, as you pointed out, returning 
> NFS4ERR_DELAY is not a legal error. What would be your suggestion for 
> an error code that a server could return to the client for a RENEW op 
> while the file system transition is in play? NFS4_OK is one of the 
> choices that the implementors had considered, which I think may work 
> out as well.
>
>
> migration, once the state migration completes, a server could hold on 
> to the lease till it sees a state renewing op from the client (with 
> certain upper bounds of course), at which point the server will stop 
> sending NFS4ERR_DELAY to the client for any state renewing op, and the 
> client can resume regular business.
>
> NFS4_OK (just  yet) in the specific case of RENEW, except it sure 
> would be nice to have universal "stalling" op.
>
>
> *From:[email-masked] <mailto:[email-masked]> 
> [mailto:[email-masked]] *On Behalf Of *Noveck, David
> *Cc:* [email-masked] <mailto:[email-masked]>; nfsv4 list 
> ([email-masked] <mailto:[email-masked]>)

---

From: "Myklebust, Trond" <[email-masked]>
Date: Mon, 8 Jul 2013 21:37:15 +0000

Right, but my point is that doing so in the particular case of RENEW is bad as it leaves the lease timeout completely unanchored.

From: Piyush Shivam [mailto:[email-masked]]
Sent: Monday, July 08, 2013 5:34 PM
To: nfsv4 list ([email-masked])
Cc: Myklebust, Trond; Noveck, David; [email-masked]
Subject: Re: [nfsv4] draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

Just to be clear, I am not making a case for returning NFS4ERR_DELAY on RENEWs, but rather looking for _a_ legal and interoperable way to stall the client for _all_ possible ops from the client while the file system transition is in play. Clearly, as you pointed out, returning NFS4ERR_DELAY for RENEW sure sounds awkward, especially in the general case.

On 07/ 8/13 03:55 PM, Myklebust, Trond wrote:
Returning NFS4ERR_DELAY on RENEW sounds like a recipe for disaster. I can’t see why you’d even want to begin considering to start contemplating it…
The larger question in play is: how do we "stall" the client while the file system transition is in play. In general the strategy has been to return NFS4ERR_DELAY, but that does not work well for ops where NFS4ERR_DELAY is not a legal error. What would be your suggestion for an error code that a server could return to the client for a RENEW op while the file system transition is in play? NFS4_OK is one of the choices that the implementors had considered, which I think may work out as well.

How is the client supposed to know when it needs to retry in order to avoid losing the lease? In most cases, we send RENEW when we’re close to the _end_ of the lease period…
A client implementation could choose to continue sending RENEWs at its usual cadence, because,  RENEW has an inbuilt notion of delay (e.g., close to end of the lease). In the special case of file system migration, once the state migration completes, a server could hold on to the lease till it sees a state renewing op from the client (with certain upper bounds of course), at which point the server will stop sending NFS4ERR_DELAY to the client for any state renewing op, and the client can resume regular business.

That being said, I don't see an advantage of using NFS4ERR_DELAY over NFS4_OK (just  yet) in the specific case of RENEW, except it sure would be nice to have universal "stalling" op.

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of Noveck, David
Sent: Monday, July 08, 2013 4:08 PM
To: Piyush Shivam
Cc: [email-masked]<mailto:[email-masked]>; nfsv4 list ([email-masked]<mailto:[email-masked]>)
Subject: Re: [nfsv4] draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

From: Piyush Shivam [mailto:[email-masked]]
Sent: Monday, July 08, 2013 3:48 PM
To: Noveck, David
Cc: Chuck Lever ([email-masked]<mailto:[email-masked]>); [email-masked]<mailto:[email-masked]>; nfsv4 list ([email-masked]<mailto:[email-masked]>)
Subject: Re: draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

Thanks,
Piyush

---

From: "Myklebust, Trond" <[email-masked]>
Date: Mon, 8 Jul 2013 20:55:48 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Noveck, David
Sent: Monday, July 08, 2013 4:08 PM
To: Piyush Shivam
Cc: [email-masked]; nfsv4 list ([email-masked])
Subject: Re: [nfsv4] draft-ietf-nfvs4-rfc3530-migration update-01 and forthcoming versions.

Thanks,
Piyush