Subject: Re: [nfsv4] Proposal for adding reclaim bit to pNFS COMMITLAYOUT

From: [email-masked]
Date: Fri, 27 Jan 2006 17:31:01 -0500

bfields at fieldses.org wrote:
> On Fri, Jan 27, 2006 at 12:42:13PM -0500, fridella_stephen at emc.com
wrote:
> > One other point---isn't this already a requirement of NFSv4? During
> > the grace period, aren't writes disallowed due to potential conflicts
> > with locks that have yet to be reclaimed?
>
> I think that should only be the case with special-stateid-writes or
> writes to files that may have mandatory locking in force.
>
> If a write is performed with a "real" stateid (derived from an open)
> then we know it can't conflict with a still-to-be-reclaimed open
> (otherwise the open couldn't legitimately have been reclaimed), so the
> only possible conflict is with a mandatory lock.
> --b. 

In light of these comments I went back to the NFSv4 RFC and
carefully re-read the recovery section (hmmm... maybe I should
have done this before my original post, huh?  ;) )  The RFC takes
the attitude:  "If the server can guarantee that a read or write
does not conflict with any potential un-reclaimed lock, it can
honor the request during the grace period."

So if we take the same approach here, then depending on the storage
protocol used, the meta-data server can allow standard NFSv4
writes to proceed during the grace period---as long as it can
guarantee that the write will not conflict/interfere with some
un-reclaimed provisional allocation for another file's layout.

As a practical matter, block/volume servers which don't support
persistent provisional allocation would probably just reject
write requests during the grace period for any file that may
have un-reclaimed layouts.  File and Object servers may be able
to provide the required guarantee (since allocation generally
happens at the storage server) and start accepting write requests
earlier.

So I propose the following text:

change:

"Note that the
meta-data server will not allow any operations that could potentially change
the state of the file-system during the recovery grace period. Thus getting
a new pNFS layout and writing of dirty data must wait until the recovery
grace period is over."

to:

"Note that during the recovery grace period, the meta-data server cannot
allow operations that could potentially conflict with uncommitted
layout changes that a client may yet attempt to reclaim.  Some meta-data
servers (e.g. those that persistently save the provisional allocation
state for each file, or those where the allocation for each file is
handled by the data storage device) may be able to guarantee for a
particular WRITE or GETLAYOUT request, that there is no way that the
request will interfere with some client's attempt to reclaim uncommitted
layout changes.  These servers are free to service such requests during
the recovery grace period.  If the meta-data server cannot make such
a guarantee for a request, it must reject the request with the status
NFS4ERR_GRACE."

Make sense to everybody?
-Steve

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Spencer Shepler <[email-masked]>
Date: Fri, 27 Jan 2006 16:42:53 -0600

> a guarantee for a request, it must reject the request with the status

Note to Steve: I am not harrassing you about this but...

I would like to make two suggestions related to proposing text
destined for the RFC.  The first is to make the sentences short, clear,
and use simple words (remember that english will not be the first language
for many readers).

Second is that I believe we need to, in the construction, make 
judicious use of the MUST, SHOULD, MAY phrasings.

In the case above, given the gravity of the situation, it may be good
to start with: "..., the meta-data server MUST NOT allow operations
that may conflict with uncommitted layout changes..."

The reason I may this general suggestion is that it will draw out the
fact that this is very important and will draw our collective attention
to the detail such that it receives appropriate review.

Your editor,
Spencer

---

From: "J. Bruce Fields" <[email-masked]>
Date: Fri, 27 Jan 2006 18:07:50 -0500

> and use simple words (remember that english will not be the first language

According to my copy of the rfc index, rfc3530 is currently the *second*
longest rfc by number of bytes (after rfc 3261, SIP).

Second is clearly not acceptable.  If we're going to seize (and
maintain) the lead, then we need longer words and sentences wherever
possible.

--b.

---

From: Spencer Shepler <[email-masked]>
Date: Fri, 27 Jan 2006 17:20:55 -0600

> maintain) the lead, then we need longer words and sentences wherever

I agree; we must be first.  However, given the number of things we would
like to have in 4.1, meeting the byte requirements should not be
a problem.

Spencer