Subject: [nfsv4] Copy Offload Questions

From: Bryan Schumaker <[email-masked]>
Date: Thu, 25 Jul 2013 10:27:25 -0400

Hi,

I have a Linux prototype of the COPY operation, but I need a few clarifications about the spec.

1) Why does CB_OFFLOAD need a stateid and a filehandle to represent the transaction?  Shouldn't a stateid alone be enough?

2) The WRITE operation uses a count4 (uint32) to represent the amount of data written, but COPY uses length4 (uint64).  Is this to account for large files?

~ Bryan

---

From: "Noveck, David" <[email-masked]>
Date: Thu, 25 Jul 2013 12:32:26 -0400

1) The fh is not absolutely necessary, but it is helpful to have, particularly in the case of a CB_OFFLOAD that races ahead of the response to the WRITE_PLUS or COPY.  The issuer will know the fh of file involved, but he only knows the stateid after he gets the response.

2) Yes.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Bryan Schumaker
Sent: Thursday, July 25, 2013 10:27 AM
To: [email-masked]
Subject: [nfsv4] Copy Offload Questions

Hi,

~ Bryan
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Bryan Schumaker <[email-masked]>
Date: Thu, 25 Jul 2013 16:18:29 -0400

Thanks!

Next question:  the spec says that the server should create the file if it doesn't already exist, but it also asks for ca_dst_stateid that must refer to existing open or locking states.  How would we ever get into the situation where we have a stateid for something that doesn't (yet) exist?

- Bryan

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 26 Jul 2013 04:02:37 +0000

> Next question:  the spec says that the server should create the file if it doesn't already exist, but it also asks for ca_dst_stateid that must refer to existing open or locking states.  How would…

Not captured in the spec, but I thought we agreed that it was
the responsibility of the client to get source and destination lock
state to protect the files during the copy.

This would argue that the client would first create the file, getting
the lock state, and then pass just the CURRENT_FH and
a ca_destination of zero length.

On the other hand, as you point out, if the client uses
a ca_destination of non-zero length, it cannot pass in a
meaningful ca_dst_stateid if it expects the destination
to create that file.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 26 Jul 2013 15:53:07 +0000

> meaningful ca_dst_stateid if it expects the destination

The problem is that there seems to be a spec disconnect between the need
to pass in a lock stateid argument (which is described as a MUST), and
the "create destination file" feature in copy offload.

I'd be quite happy to get rid of the "create file" feature. I can't see
that it adds much in the way of functionality to the protocol, and the
performance gains that it provides are likely to be negligeable when
copying large files.

Cheers,
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: "Haynes, Tom" <[email-masked]>
Date: Fri, 26 Jul 2013 19:55:03 +0000

> performance gains that it provides are likely to be negligeable when

My guess is that the create functionality was included to support non-NFSv4
based copy engines that wanted to do a minimal set of NFSv4.2 operations
to set up the offload.

I.e., say the offload engine is "ftp", it does not work with an open FH unless
we are forcing the destination server to use a non-standard version of
ftp.

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 26 Jul 2013 20:16:45 +0000

> we are forcing the destination server to use a non-standard version of

Well... I'm not really seeing how ftp could really be used for any of
this, given that it also doesn't have the filehandle support that you
need for the create operation. Are there other protocols that would be
better candidates for this?

Either way, what is the killer application for doing the file create
+copy offload as one operation?

NetApp

---

From: "McDonald, Alex" <[email-masked]>
Date: Sat, 27 Jul 2013 07:02:34 +0000

Cloud based object operations, where the copy offload is best to be atomic; makes RESTless requests possible. 
  
Sent on my bb

----- Original Message -----
From: Myklebust, Trond [mailto:[email-masked]]
Sent: Friday, July 26, 2013 01:16 PM
To: Haynes, Tom
Cc: [email-masked] list <[email-masked]>
Subject: Re: [nfsv4] Copy Offload Questions

NetApp

---

From: "Haynes, Tom" <[email-masked]>
Date: Sat, 27 Jul 2013 18:04:25 +0000

> Cloud based object operations, where the copy offload is best to be atomic; makes RESTless requests possible. 

You, yes you laddie, how can you have any locking, when you don't open your file?

You've got to do the rest of the song yourself!

If we have to support such scenarios, then what do we put in for the stateid?

If we are using native NFSv4.2 copy offload, then the file will have to be
opened first to protect it and to help in network partitioning. So we have the
stateid and we would not be creating the file in this operation.

If we are using anything else that depends on creating on the fly, then the
stateid is unknown. We could define a special one.

---

From: "Haynes, Tom" <[email-masked]>
Date: Mon, 29 Jul 2013 11:29:41 +0000

I wanted to back up - the original draft by Lentini, et al., did not have the stateids. We
added that to address gaps in protecting the files.

The uses cases in that draft were focused on making sure we did not mandate
an inter-server copy protocol. I think that led to trying to make COPY a
swiss army knife, i.e., it combines OPEN and WRITE.

The ca_destination semantics allow for WRITE and OPEN, but the returned
results only allow for WRITE.

While we could fix COPY to not allow creating a file, we have to examine whether
that violates the requirements about specifying the copy protocol. The answer is
that it does not. It is not the ca_destination semantics which dictates the
copy protocol, it is the ca_source_server which does that.

As Alex points out, all the current ca_destination facilitates is reducing the
number of operations in the compound, i.e.,  atomicity. While we could
continue to allow that, we would need to beef up the COPY results to include
those already in the OPEN results. 

It is simpler to remove ca_destination altogether and treat the COPY operation
as a WRITE.