Subject: Re: [nfsv4] 3530bis Issue 39: Clarification on renewing sequence IDs

From: Robert Thurlow <[email-masked]>
Date: Thu, 04 Nov 2010 15:57:05 -0600

Robert Thurlow wrote:
> Hi folks,
> 
> This is issue 39 from 
> http://github.com/loghyr/3530bis/blob/master/tasklist.txt.
> 
> In implementing NFSv4 migration support, we believe that
> MOVED and LEASE_MOVED need to be added to the list of errors
> in 8.1.5 which do NOT result in incrementing the open owner
> or lock owner sequence ID.  The goal is to make the sequence
> ID readily calculable for both the client and the destination
> server after the migration has occurred.
> 
> On the 3530bis call, this appeared exactly backwards to some
> others - that since a completely gross error had not occurred,
> we should increment the sequence ID and the client and the
> destination server should know to expect that when they interact
> after a migration.  I do not know this issue well enough to
> properly defend a position, so please reply with your reasoned
> opinion :-)

I don't think this has had a response.  If you disagree with
the wording change, now is the time to say so.

Rob T

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 05 Nov 2010 09:59:14 -0400

> the wording change, now is the time to say so.

As stated on the confcall, I strongly disagree with this change w.r.t.
NFS4ERR_LEASE_MOVED. The operation that resulted in a
NFS4ERR_LEASE_MOVED cannot be safely replayed if the sequence id has not
been bumped.

The point is that NFS4ERR_LEASE_MOVED is an error that depends on the
state of a _different_ filesystem. It does not even pertain to the
actual state you are trying to modify (and is a monstrous hack). Worse
yet, that error condition can be cleared at any time with no
consequences for the stateids held by the client, so unlike
NFS4ERR_BAD_STATEID or NFS4ERR_BAD_SEQID, there is no ordering w.r.t.
the operation that you are retrying.

IOW: if the error condition happens to get cleared between two replays
of the operation, the client may end up getting 2 conflicting replies
(one NFS4ERR_LEASE_MOVED, the other being a change of state on the
server). Which one does it choose?

So how about counter-proposal: we bury NFS4ERR_LEASE_MOVED at a
cross-roads with a stake through its heart, and promise never to mention
it again except when the kids need scaring to bed...

Trond

---

From: <[email-masked]>
Date: Sun, 7 Nov 2010 01:04:11 -0500

I agree with Trond's argument as to seqid, i.e. that you should
increment the seqid in the case of NFS4ERR_LEASE_MOVED.

NFS4ERR_LEASE_MOVED was buried at a cross-roads in RFC5661, but there we
have SEQ_STATUS_LEASE_MOVED so we don't need it.

If you bury it in RFC3530bis, you do have to have some way to deal with
the issue of letting the client find out that there is a migrated lease.
Otherwise, there is an unbounded period in which the new server will not
hear from the client and the client's open files could be lost.

The alternative to the monstrous hack would be to require the server to
simulate a reboot.  The client would see a STALE client or stateid error
and then he would go through the reclaim sequence for both any migrated
and non-migrated fs's.   That seems harder to make happen than
LEASE_MOVED, as monstrous as it is.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Trond Myklebust
Sent: Friday, November 05, 2010 9:59 AM
To: Robert Thurlow
Cc: NFSv4
Subject: Re: [nfsv4] 3530bis Issue 39: Clarification on renewing
sequence IDs

Trond
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 18 Nov 2010 12:09:03 -0500

So, I don't think that SEQ_STATUS_LEASE_MOVED should be the last word on
this process.

SEQ_STATUS_LEASE_MOVED has the same scalability problems that
NFS4ERR_LEASE_MOVED does: if the server is exporting 1000 or so volumes,
and one of them moves, it has to check every time a client sends a
SEQUENCE op (i.e. pretty much every COMPOUND) if that client has
completed its search for the absent volume. Furthermore, it will be
receiving a bunch of new RPC requests as the clients all probe 1000
volumes for the absent one.

NFSv4.1 has solved the problem of callbacks by means of the session
backchannel. Why can't we now solve the problem of LEASE_MOVED by adding
a CB_LEASE_MOVED operation which reports not only that a filesystem is
missing, but also identifies which filesystem (e.g. by including the
fsid)?

Cheers
  Trond

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: <[email-masked]>
Date: Fri, 19 Nov 2010 17:39:06 -0500

SEQ_STATUS_LEASE_MOVED was not intended to be the last word on this
process.  It was only intended to be the last word on it in NFSv4.1 :-)

I don't see the scalability issue for the client SEQUENCE check.  You
have a per-client data structure and you keep a set of status bits in it
which you or into every sequence response.  There is also a count of
file systems that need to get their locations fetched.  You only change
the count when people get locations or something moves.  When the count
changes to or from zero the status bit changes.

I do see the issue for client probe.  I think CB_LEASE_MOVED is a fine
solution.  I think we may have been in this mode of being reluctant to
make it a requirement to have callbacks working.  I think we don't have
to require it.  You can still have SEQ_STATUS_LEASE_MOVED.  It is just
that you now have a simple way to find out the specific fs which moved
and that it will short-circuit the process of finding the fs that moved
so that you can get the fs_locations[_info] and thus resetting the
status bit.

I think that this kind of thing makes sense for v4.2, no matter what
scope decision we make.  Even if you were to decide on a bug-only minor
version, I think this kind of cleanup makes sense to put in the minor
version.  Let's not get into is-this-*really*-a-bug mode.  I accept the
onus of having co-edited a buggy minor version.  I can't see us doing a
critical-bug-only minor version.    

I think we will need to have a v4.2 document minor version document in
place soon.  This kind of small thing (bug or not) is just not big
enough to merit an internet-draft.  If you require it, these things will
just not get done.  I think that once we have a framework for this sort
of small addition, we can publish the text for the new op or callback in
messages on the working group list and then incorporate them in the next
v4.2 draft, once there is rough consensus.  My suggestion is that we
make it the practice to publish proposed v4.2 text in the form of
xml2rfc xml within the working group email to make it simpler for the
v4.2 document editor to incorporate.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Trond Myklebust
Sent: Thursday, November 18, 2010 12:09 PM
To: Noveck, David
Cc: [email-masked]; [email-masked]
Subject: Re: [nfsv4] 3530bis Issue 39: Clarification on renewing
sequence IDs

Cheers
  Trond

NetApp

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 19 Nov 2010 17:58:53 -0500

> v4.2 document editor to incorporate.

Tom, now is your chance to shine even brighter! Another git repository
to complement git://github.com/loghyr/3530bis.git and a 4.2 review
process incorporating Dave's suggestions?

Cheers
  Trond

NetApp

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 19 Nov 2010 17:49:19 -0800

Yeah, I was planning on starting it anyway. I'll get it done by Monday.

And github works great for me.

---

From: "Spencer Shepler" <[email-masked]>
Date: Fri, 19 Nov 2010 21:41:19 -0800

While publishing additions to the WG alias in xml is fine
with me, please ensure that it is somewhat readable.

Also, take care not to use github as the publishing mechanism.  I-Ds are
the place to capture reasonable interim versions of the I-D.

Spencer

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Haynes, Tom
> Sent: Friday, November 19, 2010 5:49 PM
> To: Trond Myklebust
> Subject: Re: [nfsv4] 3530bis Issue 39: Clarification on renewing sequence
> IDs
> 
> >> I think we will need to have a v4.2 document minor version document
> >> in place soon.  This kind of small thing (bug or not) is just not big
> >> enough to merit an internet-draft.  If you require it, these things
> >> will just not get done.  I think that once we have a framework for
> >> this sort of small addition, we can publish the text for the new op
> >> or callback in messages on the working group list and then
> >> incorporate them in the next

---

From: Thomas Haynes <[email-masked]>
Date: Fri, 19 Nov 2010 22:33:16 -0800

> the place to capture reasonable interim versions of the I-D.

Agreed, just like the 4.1 editors used the nfsv4-editor site to maintain the
source and published the drafts as an I-D. What I did with rfc3530 was to
publish the draft first as an I-D and then to check in a copy of the text file
to github. This allowed the make process to be able to generate consistent
diffs against the published versions.

Without thinking too much, I envision the first draft to be submitted before the
interim meeting and the second draft in time for IETF 80 in Prague.

And just like the 3530bis work, the proper readable format for the xml 
is in the framework adopted for the NFSv4.1 editing process. I have no
plans to deviate from what has been working for us.

https://github.com/loghyr/NFSv4.2

is the initial port of the NFSv4.1 xml to the NFSv4.2 xml. As the README
clearly states, only the XDR xml is currently making. I'll start putting the
framework in place for the specification draft this weekend.