Subject: RE: [nfsv4] Block Layout and CB_SIZECHANGED

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 14 Jul 2006 10:11:48 -0400

> In NFSv4.1 all applications are required use some form of locking 
> in the above scenario, or else they are not guaranteed cache 
> coherency. IOW: those applications are not guaranteed "correctness" 
> even in the case of no pnfs.

But there is no precise equivalent of David's case in the no-pnfs
world.

If I put blocks beyond eof in my cache and then somebody truncates
the file, then cached writes are no invalidated.  I think this is
the case you are referring to.

But David is not talking about cached writes but writes done to 
the data server which have not been LAYOUTCOMMITed.  There is no 
non-pnfs equivalent of that.

The closest I can come is unstable writes done to the server
which have not been COMMITed.  In this case a truncate is effective
without locking.  You do the the COMMIT and the file not extended.

How you judge this case depends on what analogies you make.  Is
writing to the data server more like putting things in your cache
or it more like doing an unstable write?  I'd argue that the latter
is a more appropriate analogy.

-----Original Message-----
From: Trond Myklebust [mailto:[email-masked]] 
Sent: Friday, July 14, 2006 8:13 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [nfsv4] Block Layout and CB_SIZECHANGED

On Thu, 2006-07-13 at 23:15 -0400, [email-masked] wrote:
> >   - People are still confused about what it does in the block's
case.
> >     David Black gave an example that helped some understand, others
not.
> >     I think it needs to be made very clear how this is not an
> >     optimization but rather required for correctness.
> 
> For blocks, the need for CB_SIZECHANGED is caused by lazy evaluation 
> of COMMIT-based EOF changes.  The sequence of operations is the
> following:
> 
> 1) pNFS client takes out an extent from 32k to 64k, and writes data.
> 	It marks the written area as needing to be COMMIT-ed, but
> 	doesn't do the COMMIT.
> 2) Some other client uses SETATTR to truncates the file to be 4k in
size.
> 
> The file is now 4k in size, but if the pNFS client does its COMMIT, 
> the file will erroneously grow to 64k in size.  Therefore, as part of 
> the SETATTR, the pNFS client has to be told about the new file size.  
> The client abandons its commit for 32k-64k as a result of this 
> notification, and it *has* to be told about the new file size in order

> to do so.
> 
> That's why at least the SETATTR case of CB_SIZECHANGED is needed for 
> correctness.

In NFSv4.1 all applications are required use some form of locking in the
above scenario, or else they are not guaranteed cache coherency. IOW:
those applications are not guaranteed "correctness" even in the case of
no pnfs.

Now at the IETF meeting, you appeared to imply that there was some issue
with the layout itself being modified by a SETATTR call (something about
a change of the state of the blocks "owned" by the client(?)). Could you
please explain again what it is about the layout that changes and why it
is that the client needs to be notified.

Cheers,
  Trond

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 14 Jul 2006 10:27:59 -0400

> is a more appropriate analogy.

I'd argue that until you commit the layout, you are still in the
situation where the data has not been written. You have not done the
equivalent of a full NFSv4.0 unstable WRITE since a successful unstable
write must update both the data _and_ the metadata in the server's
cache.
IOW the point at which the written data becomes visible to others is
what matters, and that means after LAYOUTCOMMIT.

Cheers,
  Trond