Subject: [nfsv4] A few questions about CLONE and COPY in draft-ietf-nfs4-minorversion2-36

From: Trond Myklebust <[email-masked]>
Date: Thu, 23 Apr 2015 19:25:33 -0400

1) If the cl_src_offset and/or cl_dst_offset are not aligned to the
clone block size, then what is the return value/protocol behaviour for
CLONE? Should it return NFS4ERR_BAD_RANGE?

2) Shouldn't CLONE be able to return NFS4ERR_LOCKED if the server
supports mandatory locking?

3) Shouldn't CLONE be able to return NFS4ERR_DQUOT?

4) Shouldn't CLONE and COPY be able to return NFS4ERR_XDEV for the
same reasons that LINK and RENAME do (i.e. crossing a filesystem
and/or security policy boundary)?

5) What happens to the current stateid? (this question goes for
COPY_OFFLOAD too, by the way) These 2 operations are the only ones
that take 2 stateids, so I don't think there is any precedent in
RFC5661 or RFC7530 to point to.

6) Do we need to describe what happens to the current filehandle?
There is a precedent here in the form of LINK, and RENAME AFAIK.

7) Shouldn't NFS4ERR_METADATA_NOTSUPP be removed from Section 11.2?

Cheers
  Trond

---

From: David Noveck <[email-masked]>
Date: Thu, 23 Apr 2015 20:29:44 -0400

1)  I think people have been assuming INVAL for this.  Given the definition
of BAD_RANGE in RFC5661, it isn't a good fit.

2) Yes.

3) Yes.

4) Yes for CLONE.  I don't see a reason to do this for COPY.

5) Nothing, since these operations do not return a stateid.  IIRC, current
stateid is not changed simply because you pass a stateid (or two) to an
operation.

6)  I think the default assumption is that current and saved filehandle
stay as they are.

7) Yes.

On Thu, Apr 23, 2015 at 7:25 PM, Trond Myklebust <[email-masked]> wrote:

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Trond Myklebust <[email-masked]>
Date: Thu, 23 Apr 2015 21:09:23 -0400

> of BAD_RANGE in RFC5661, it isn't a good fit.

OK, then let me make a case for still allowing BAD_RANGE, but for a
different reason: I'm aware of at least one proposed implementation of
copy offload that does not allow 64-bit ranges to be copied; instead
they would need to be chunked into 32-bit ranges. Would this be a more
appropriate fit?

> 4) Yes for CLONE.  I don't see a reason to do this for COPY.

Isn't the same underlying mechanism that is used for CLONE also a
valid implementation of COPY?

> 6)  I think the default assumption is that current and saved filehandle stay
> as they are.
>
> 7) Yes.
>

---

From: David Noveck <[email-masked]>
Date: Fri, 24 Apr 2015 06:20:14 -0400

> I'm aware of at least one proposed implementation of

It would be, but I imagine that some people on the client side of things
might
argue that it would be more appropriate to put the burden of adapting to
this
implementation limitation into the server rather than forcing alll clients
to deal
with it.

> valid implementation of COPY?

I would expect COPY implementations to use CLONE facilities when they are
available.

However, that doesn't mean that every CLONE error should be a COPY error
as well.  For example, if the offsets are not compatible with the clone
blocksize, copy will not return INVAL.  I expect that implementations would
do a COPY that does not involve COW techniques.  I would expect the
situation
with XDEV to be similar.

On Thu, Apr 23, 2015 at 9:09 PM, Trond Myklebust <[email-masked]> wrote:

> stay

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 24 Apr 2015 09:05:20 -0400

> implementation limitation into the server rather than forcing alll clients

That may not be possible here, given that CLONE is required to follow
rather stringent atomicity guarantees.

> do a COPY that does not involve COW techniques.  I would expect the

Fair enough.

---

From: Benny Halevy <[email-masked]>
Date: Fri, 24 Apr 2015 16:36:22 +0300

(Replying from my mobile phone so I apologize for top posting and html
encoding)

Regarding CLONE's atomicity requirements, what's are we requiring exactly
and is it realistic for all servers implementing CLONE to provide these
requirements as there is no other I/O operation in NFS that provides atomic
semantics?

1. Failure semantics: do we expect a CLONE op that hit a server crash
before sending a reply to either fully succeed or be rolled back?

2. How do conflicting CLONE operations (with overlapping byte ranges) need
to be handled?

3. How do conflicting READ and WRITE operation are to be handled?

4. Inter operation with LOCK ops? My expectations when adding NFS4ER_LOCKED
as an allowed error is that CLONE will fail in the face of conflicting lock
based on the stateid.

Given 4 above can the client and server negotiate atomicity based on a
boolean in the request and reply and if the server does not support atomic
CLONE have the client use byte range locking to achieve the atomic
semantics?

Benny

---

From: Trond Myklebust <[email-masked]>
Date: Fri, 24 Apr 2015 09:49:12 -0400

On Fri, Apr 24, 2015 at 9:36 AM, Benny Halevy <[email-masked]> wrote:
> 1. Failure semantics: do we expect a CLONE op that hit a server crash before
> sending a reply to either fully succeed or be rolled back?
>

This is where the atomicity requirement is useful. It deals with (1),
(2) and (3) without any room for ambiguity.

> CLONE have the client use byte range locking to achieve the atomic

I'm fine with CLONE and COPY having different requirements around
atomicity. It seems to me that if you can't implement CLONE
atomically, then you might still be able to implement COPY.

My only question was around whether or not a 32-bit version of CLONE
is allowed, and whether or not it needs an error message to inform the
client. I guess the alternative is NFS4ERR_NOTSUPP, and proceeding to
implement COPY instead.

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 25 Apr 2015 15:29:49 +0200

> client. I guess the alternative is NFS4ERR_NOTSUPP, and proceeding to

What error would you want to return?  Either way the only sensible
things to communicate this is to have a max_clone_size attribute.

Does you limited clone implementation still support full file clones
when the file size exceeds 32 bits?

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 25 Apr 2015 15:31:49 +0200

> and/or security policy boundary)?

Both already are in the valid error list for CLONE.

---

From: David Noveck <[email-masked]>
Date: Sun, 26 Apr 2015 06:43:44 -0400

> client.

I think the issue is whether the client could do anything useful with
such an i-can't-handle-64bit-ranges indication.

Benny recently raised the issue of whether byte-range locking could help
the client provide atomicity;

   - If all clients used advisory locking around the clone operation, that
   would address atomicity requirement 2.
   - If the server supported mandatory byte-range locking that would
   also address atomicity requirement 3.
   - There doesn't seem to be any way to address atomicity requirement 1.

> I guess the alternative is NFS4ERR_NOTSUPP,

If 1, 2, and 3 are indeed CLONE requirements that clients care about, it
seems that a server that cannot provide those requirements doesn't really
support CLONE.  I don't think it make sense to carve out an exception for
ranges over 4GB.

> and proceeding to implement COPY instead.

But that doesn't give you support for any atomicity requirements at
all. However, if you surrounded it by
byte-range locking as Benny suggests, it could give support for requirement
2 and possibly requirement 3.

So you are no better off than stringing together sub-4GB CLONE operations,
from an atomicity point point of view.  I can see that some clients might
choose COPY as simpler in this case, however.  Byte-range locking could be
used if the 2/3 of atomicity that is possible is important to that
particular client.  You could also provide that same degree of atomicity by
 using share reservations, although this might be too coarse-grained for
some people's tastes.

On Fri, Apr 24, 2015 at 9:49 AM, Trond Myklebust <[email-masked]> wrote:

> before
> >> >> > stateid is not changed simply because you pass a stateid (or two)
> to
> >> >> > an
> >> >> > operation.
> >> >> >
> >> >> > On Thu, Apr 23, 2015 at 7:25 PM, Trond Myklebust <
> [email-masked]>
> >> >> > wrote:
> >> >> >>

---

From: Christoph Hellwig <[email-masked]>
Date: Mon, 27 Apr 2015 15:38:29 +0200

> such an i-can't-handle-64bit-ranges indication.

I don't think a special indicator is very helpful, but even without it
an implementation that can only clone 32-bit ranges can be useful IFF
it also supports full file clones.  On the other hand I'd be really curious
why an implementation would have this limit.

---

From: Trond Myklebust <[email-masked]>
Date: Mon, 27 Apr 2015 10:39:50 -0400

> why an implementation would have this limit.

The proposed copy offload system calls on Linux have been flirting
with using a size_t for the length. I'm trying to change that proposal
to use offset ranges instead of a length, but it does raise issues...

Cheers
  Trond