Subject: [nfsv4] NFSv4.2 INITIALIZE vs. NFS WRITE

From: Chuck Lever <[email-masked]>
Date: Mon, 23 Jul 2012 12:09:41 -0400

I've been struggling with the architecture of the proposed NFSv4.2 INITIALIZE operation.  Here are a few important differences between a traditional NFS write operation and the new INITIALIZE operation.

 o  Bytes affected by a single operation:  A single WRITE operation is always limited to no more than "wsize" bytes.  There is no cap on "wsize" in the NFS protocol, but on common server implementations, this is typically no more than a megabyte.  A single INITIALIZE(PLAIN_HOLE) or INITIALIZE(APP_HOLE) can potentially alter as many bytes as the largest possible file can hold.  The protocol allows file sizes up to on the order of 2 ^ 63 bytes, though common server implementations have a somewhat lower limit on file size.

 o  Synchronicity: An NFS WRITE operation allows UNSTABLE, DATA_SYNC, or FILE_SYNC writes.  WRITE(UNSTABLE) or WRITE(DATA_SYNC) requires a subsequent COMMIT operation, because making written data durable is potentially a slow physical operation.  INITIALIZE, as it is described now, does not support the notion of unstable file changes, and never requires a COMMIT.

The lack of control over synchronicity becomes interesting because an INITIALIZE operation can affect many more bytes than can a WRITE operation.  I feel it would be appropriate to allow an asynchronous arrangement for INITIALIZE, similar to the proposed server-to-server copy feature in NFSv4.2.  COMMIT'ing a relatively small amount of data is expected to occur easily within an NFS retransmission timeout, but updating a sufficiently large file via INITIALIZE may be more than a single COMMIT can handle without triggering retransmit timeouts.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

---

From: Rick Macklem <[email-masked]>
Date: Mon, 23 Jul 2012 18:32:56 -0400 (EDT)

Chuck Level wrote:
> I've been struggling with the architecture of the proposed NFSv4.2
> INITIALIZE operation. Here are a few important differences between a
> traditional NFS write operation and the new INITIALIZE operation.
> 
> o Bytes affected by a single operation: A single WRITE operation is
> always limited to no more than "wsize" bytes. There is no cap on
> "wsize" in the NFS protocol, but on common server implementations,
> this is typically no more than a megabyte.

I had (maybe mistakenly) thought that the "maxwrite" attribute set
the upper bound on the size of a write? (Although I can see an argument
that it was only a "recommendation" and isn't a mandatory attribute.)

rick

> A single
> INITIALIZE(PLAIN_HOLE) or INITIALIZE(APP_HOLE) can potentially alter
> as many bytes as the largest possible file can hold. The protocol
> allows file sizes up to on the order of 2 ^ 63 bytes, though common
> server implementations have a somewhat lower limit on file size.
> 
> o Synchronicity: An NFS WRITE operation allows UNSTABLE, DATA_SYNC, or
> FILE_SYNC writes. WRITE(UNSTABLE) or WRITE(DATA_SYNC) requires a
> subsequent COMMIT operation, because making written data durable is
> potentially a slow physical operation. INITIALIZE, as it is described
> now, does not support the notion of unstable file changes, and never
> requires a COMMIT.
> 
> The lack of control over synchronicity becomes interesting because an
> INITIALIZE operation can affect many more bytes than can a WRITE
> operation. I feel it would be appropriate to allow an asynchronous
> arrangement for INITIALIZE, similar to the proposed server-to-server
> copy feature in NFSv4.2. COMMIT'ing a relatively small amount of data
> is expected to occur easily within an NFS retransmission timeout, but
> updating a sufficiently large file via INITIALIZE may be more than a
> single COMMIT can handle without triggering retransmit timeouts.
> 
I think that, if asynchronous INITIALIZE is allowed, there should be a
STABLE/UNSTABLE request/reply arg, so that servers can implement STABLE
if they can easily do so.
But, I'm not strongly opinionated w.r.t. this, rick

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4