Subject: RE: [NFSv4] Hints for client on LAYOUTRECALL

From: [email-masked]
Date: Thu, 10 Aug 2006 15:35:08 -0400

So are you saying that, in general, a client that receives a
LAYOUTRECALL should not flush dirty data before returning the
LAYOUT?  Or are you saying that we should not recall a layout
from a client on truncate?

If it's the former, then I think that changes a few of my personal
assumptions about how caching and layouts interact, and I have
some questions (see below).

If it's the latter, unfortunately, the block/volume server has
no choice.  When a file is truncated, the blocks are freed.  If
a client has a layout which covers these blocks, the layout becomes
semantically invalid.  The server MUST recall the layout before
truncating the file.  The question becomes: what should the client
do upon receiving this recall?

On Wed, 09 Aug 2006 10:15:23 -0400, Trond Myklebust wrote:
> On Wed, 2006-08-09 at 09:59 -0400, fridella_stephen at emc.com wrote:
> > Ok I see... so in the NFS caching model, a truncate from
> > client A should not effect dirty data cached at client B.
> 
> Exactly. The possible exception to this rule is if client A holds a
> delegation. If so, then the act of truncating the data will cause the
> delegation to be recalled, at which point the client may choose to flush
> its data (which will subsequently be lost).
> Strictly speaking, though, that client behaviour is a violation of
> close-to-open semantics.
> 

Is this actually a violation?  My understanding of open-to-close
semantics is that it requires the data to be flushed before a close
succeeds.  This doesn't eliminate the possibility that the client
can pro-actively flush dirty data, does it?  So any application that
is expecting open-to-close semantics cannot rely on an assumption
that data will not be flushed until a "close" is called.

By the way, fwiw, this delegation recall model is the same mental
model I have personally been using for layout reccall in pNFS--->
client flushes data, client commits layout changes, client returns layout.
Does this match everybody else's understanding, or am I way off the mark?

-Steve

> > Just so I'm clear, does this mean the behavior where the
> > block/volume pNFS server recalls the layout from B when
> > it receives the truncate from A, and B flushes its dirty data,
> > then commits its layout, and finally the file is truncated
> > on the server, is also incorrect?
> > 
> > It seems that way, since in this case, as well as the clobber
> > case, the application on client B will lose its cached data
> > before it calls close...
> > -Steve
> 
> Yes.
> 
> I think the situation is slightly more hairy when it comes to a READ
> layout. In that case, the layout probably should be recalled, since the
> client would normally expect that subsequent READs to the metadata
> server will see the effect of the truncation.
> The close-to-open cache consistency model does not actually require this
> either, though, since clients are supposed to avoid changing the file
> while there are other readers.
> 
> Cheers,
>  Trond 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 10 Aug 2006 16:27:14 -0400

> from a client on truncate?

LAYOUTRECALL should definitely not require the client to flush any
cached dirty data. It is _not_ an instrument for managing caching.

I'd advocate recalling any READ layouts for clients on truncate, but I
don't see why we should need to recall WRITE layouts.

> do upon receiving this recall?

If the layout is for WRITE only, why would you need to free the blocks?
The client has already asserted that it wants to write to them.

Anyhow, if you do, as I said, then the client does not need to flush its
data caches.

> that data will not be flushed until a "close" is called.

The close-to-open violation consists in client A deciding to truncate
the file while client B still has it open.
That cannot happen in a strict close-to-open scenario, and so if client
A is doing this without the aid of some other locking protocol to
mediate file access with client B, then the resulting effect on the file
data is undefined.

> Does this match everybody else's understanding, or am I way off the mark?

I repeat: layout recall is _not_ to be treated by the client as a
request to start flushing its cache.

The minimal behaviour should be that the client declare the data that it
has already written to disk to the metadata server by means of a layout
commit, and then that it return the layout.
If the client is already in the process of flushing data to disk, then
for performance reasons, it may decide to complete flushing that data
before doing the layout commit and return, but it should not be forced
to do so by the rules of pNFS.

Cheers,
  Trond