Subject: [nfsv4] updated server-side copy offload I-D

From: James Lentini <[email-masked]>
Date: Mon, 6 Apr 2009 23:25:07 -0400 (EDT)

I've posted an updated NFS server-side copy offload draft. The new 
draft is available at:

 https://datatracker.ietf.org/drafts/draft-lentini-nfsv4-server-side-copy/ 

This update incorporates the feedback we have received over the past 
week and clarifies the security considerations for inter-server 
copies.

We appreciate the attention the first version received and look 
forward to feedback on this update.

-james

--
James Lentini | NetApp | 781-768-5359 | [email-masked]

---

From: Tom Haynes <[email-masked]>
Date: Mon, 06 Apr 2009 23:33:37 -0500

I really like the clarity in Section 3: Protocol Overview. It gives a 
quick synopsis of
the operations.

-------------------

Page 3:

   CB_COPY:  Used by the server to report the results of an asynchronous
      file copy to the client.

Q: Which server?

----

Page 5:

   the source server.  If NFSv4.x is used for the server-to-server copy
   protocol, the destination server can use the filehandle contained in
   the COPY request with standard NFSv4.x operations to read data from
   the source server.

Q: How does the destination server generate a stateid to be used in the
READ ops it sends to the source server?

----

Page 9:

COPY_REVOKE - still not clear that this is protected per user.

----

Pages 9-10:

COPY_REVOKE

   situation, the client may treat the copy as a failure and start
   writing to file X regardless of the ultimate status (if the network

I prefer:

   situation, the client may treat the copy as a failure and start
   writing to file X on the source server regardless of the ultimate
   status (if the network

----

Pages 9-10:

COPY_REVOKE takes a cra_destination_server to identify the destination
server. Given that the destination server may be either multihomed or
connect through a non-ethernet based protocol (and the client could
only specify it via either a name or IP), then how does the source
know which incoming requests to that FH correspond to the destination
server?

In the email discussions, we had talked about the destination server
identifying the interfaces it might use to connect. I fail to see how
that discussion is still not germane.

Further, since the destination server might already have read access
to the source server, an user on the destination might already be
reading the file via NFSv4.x. It might even be the same user name
as being used in the client initiated COPY.

How does the COPY_REVOKE differentiate these two sessions? A perfect
example of a "shared" username with different people is 'root'. If
user A on the client is doing a COPY as root@client, should they
have the right to impede user B doing a cp as root@destination?

NITS:
=====

-----

Page 5:

        | --- COPY_NOTIFY -->|                 |
        | <-- COPY_NOTIFY ---|                 |

Don't put the 'COPY_NOTIFY' on the return path. Or fix the other paths to
be the same.

I think this is most consistent:

        |---- COPY_NOTIFY -->|                 |
        |<------------------/|                 |

----

page 22:

      o User principal generates a secret it will share with the two
      servers.  This shared secret will be placed in the
      cfap_shared_secret and ctap_shared_secret of the appropriate

The text should be indented on the 2nd and 3rd line. And so on for the 
rest of items in that
list.

-----

Page 24:

   After a destination server has a "copy_to_auth" privilege established
   on it, and it receives a COPY request, if it knows it will use an ONC
...
   "copy_confirm_auth" privilege accomplishes the following:

I'd like to see this paragraph broken up into several - it is quite dense.

-----

Page 25:

   o  if a protocol like NFS is being used, with export policies, export
      policies can be over ridden in case the destination server as-an-

Is it 'over ridden' or 'over-ridden' or 'overridden'?

I think it is the last.

---

From: James Lentini <[email-masked]>
Date: Tue, 7 Apr 2009 10:00:37 -0400 (EDT)

Tom,

Thank you for the comments. Replies below:

On Mon, 6 Apr 2009, Tom Haynes wrote:

> I really like the clarity in Section 3: Protocol Overview. It gives a quick
> synopsis of
> the operations.
> 
> -------------------
> 
> Page 3:
> 

Good point. I'll note it is the destination server in the next
version.

> READ ops it sends to the source server?

With OPEN using CLAIM_FH.

> COPY_REVOKE - still not clear that this is protected per user.

Ok. I'll add further clarifications in the next version.

>   writing to file X on the source server regardless of the ultimate

Agreed.
 
> know which incoming requests to that FH correspond to the destination

COPY_REVOKE takes the same destination server as COPY_NOTICE. The 
tuple <source fh, user ID, destination's netloc> fully identifies the 
entity that was granted copy privileges on behalf of the user 
principals.

But I think the question is not about how does COPY_REVOKE match the 
destination server to the COPY_NOTIFY, but instead, when the 
destination server contacts the source server to copy, how does the 
source know for sure the destination copying is authorized?

A1. Use RPCSEC_GSSv3 with the three privileges

A2. If RPCSEC_GSSv3 is not going to be used, then we
    back to a trust model. The source server can provide
    an array of unique addresses, names, and/or URLs in the response 
to
COPY_NOTICE, and
    the client uses that array in the argument. Of course nothing
    prevents an attacker from using this same list, especially if
    RPCSEC_GSS is not being used. And if RPCSEC_GSS is being used,
    then why not use RPCSEC_GSSv3.

We could consider adding a specific "uniquifier" to the response to 
COPY_NOTICE for use with COPY, but that uniquifier would be doing the 
same thing as the shared secret in the three privileges.

A URL accomplishes the same thing, and in a manner that works with all 
protocols (at least, all those that can be specified with URLs).

So we don't think we need a third way to unambiguously identify the 
destination server.

> that discussion is still not germane.

The destination connects to the source, not the other way around. 
COPY_NOTICE now returns an array of source server netloc4s, which 
might be meaningful only behind the glass wall. The destination 
selects the source server netloc4.
 
> have the right to impede user B doing a cp as root@destination?

A normal user on a destination server that is also a conventional, 
general purpose NFSv4 client is unaffected by COPY_REVOKE. 
Effectively, in that situation, COPY_REVOKE might be a no-op, except 
that if that normal user has his privileges revoked through other 
means, such as an NFSv4 export policy change on the source, until 
COPY_REVOKE is done, the user on the client has made a grant of 
authority to read the from the source to the destination. One model to 
think of is that the general purpose NFSv4 user/client on the source 
maintains a { user credential / filehandle / destination server } 
triple that is established by COPY_NOTICE. This triple persists until 
COPY_REVOKE.

If instead, you mean what if the same client has sent COPY_NOTICE 
twice to the same source server, and sent one revoke? Good question. 
The results of EXCHANGE_ID include the server owner, and we need to 
decide whether the scope of a copy authorization is limited to the 
server owner major+minor or the server owner major (and all its 
minors). We think it should be the former (especially since RPCSEC_GSS 
handles don't span all minors of a major server owner). In that model, 
a second COPY_NOTICE for the same fh and destination server simply 
re-authorizes the destination, and a single COPY_REVOKE removes the 
authorization.

> I think this is most consistent:

Oops. That slipped by.

> The text should be indented on the 2nd and 3rd line. And so on for the rest of
> items in that
> list.

Sigh. I struggled with xml2rfc on this. I'll try to find an
incantation that generates the correct indentation in the next
version.

> I'd like to see this paragraph broken up into several - it is quite dense.

Ok.
 
> Is it 'over ridden' or 'over-ridden' or 'overridden'?

Right. overridden.

---

From: Tom Haynes <[email-masked]>
Date: Tue, 07 Apr 2009 10:21:36 -0500

> source know for sure the destination copying is authorized?

Close, it isn't authorization that I am asking about, it is identification.

And I think removing RPCSEC_GSSv3 from the picture is also helpful in 
understanding what
I am asking here.

Consider two different destination servers copying the same FH and 
neither are using the
location that the client provided in the COPY_NOTIFY.

How does the source know which of them to reject?

>     an array of unique addresses, names, and/or URLs in the response 

I don't see how this matters to stopping anything as the source may hand 
that out to
every client.

I do agree with your statement about URLs.

And now I'm popping back from the identification problem to the 
authorization one...

What I am getting to is that before you can add RPCSEC_GSSv3 as an optional
way to secure the inter-server copy, you must first define how it will 
work without
it.

That implies the "trust model" you allude to in A2. Which I think goes 
against this
on page 19:

>    o  The specification MUST NOT mandate a trust relationship between
>       the source and destination server.  The NFSv4 security model
>       requires mutual authentication between a principal on an NFS
>       client and a principal on an NFS server.  This model MUST continue
>       with the introduction of COPY.
>

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 7 Apr 2009 10:24:25 -0500

> way to secure the inter-server copy, you must first define how it will 

Acutally, you must define how this will be RPC security flavor
independent.  Which means: the destination has to give the client opaque
information that names the destination at the RPC security layer, then
the client must forward to the source so that the source can then
authorize the destination once the destination has authenticated in a
way that matches that opaque data.

This data would be:

 - IP address(es) for AUTH_SYS
 - GSS-API exported name tokens for RPCSEC_GSSv[123]

Since we're not likely to get new sec flavors not based on the GSS-API
we needn't make this data opaque -- just extensible.

Nico
--

---

From: James Lentini <[email-masked]>
Date: Tue, 7 Apr 2009 17:18:32 -0400 (EDT)

On Tue, 7 Apr 2009, Tom Haynes wrote:

> James Lentini wrote:
> > 
> > COPY_REVOKE takes the same destination server as COPY_NOTICE. The tuple
> > <source fh, user ID, destination's netloc> fully identifies the entity that
> > was granted copy privileges on behalf of the user principals.
> > 
> > destination server to the COPY_NOTIFY, but instead, when the destination
> > server contacts the source server to copy, how does the source know for sure
> > the destination copying is authorized?
> >   
> 
> And I think removing RPCSEC_GSSv3 from the picture is also helpful 
> in understanding what I am asking here.

Ok. Please keep in mind that the behavior will be different with 
RPCSEC_GSSv3 and that is the configuration we recommend.

Note that in an environment without cryptographic authentication, it 
will be impossible to securely validate an identity. If the goal is to 
guard against a malicious adversary, then real security should be 
used.

> neither are using the location that the client provided in the 
> COPY_NOTIFY.
> 
> >     an array of unique addresses, names, and/or URLs in the 
> >     response to COPY_NOTICE, and
> 
> I don't see how this matters to stopping anything as the source may 
> hand that out to every client.

Let me elaborate using your picture from earlier:

              +------- 10.100.10/24 -------------+
              |                                  |
              | 10.100.10.43                     | 10.100.10.42
         +----+---+                       +------+------+
         | Source |                       | Destination |
         +----+---+                       +------+------+
              | 10.10.20.43                      | 10.10.20.42
              |                                  |
              |                                  |
-- Glass Wall  | -------------------------------- | ----------------
              |                                  |
              |                                  |
              +------- 10.10.20/24 -+------------+
                                    |
                                    |
                                    |
                                    | 10.10.20.192
                               +----+-----+
                               | Client   |
                               +----------+

In an AUTH_SYS environment, the client will send a COPY_NOTIFY for 
10.10.20.42 to the source server.

The source server may reply to the client with a list of target 
addresses, names, and/or URLs and assign them to the unique triple: 
<source fh, user ID, destination address Y>. If the destination uses 
one of those target netlocs (these are netlocs on the source server, 
not the destination), the source server has now identified the the 
destination regardless what address the destination connects from.

For example if the source filehandle is 0x12345, the source server may 
respond to a COPY_NOTICE for destination 10.100.20.42 with the URL:

   nfs://10.100.10.43//_COPY/10.100.20.42/_FH/0x12345

When the destination contacts this URL using NFSv4.1 from any address, 
whether it is 10.100.10.42 or not, and does the following:

   COMPOUND { PUTROOTFH, LOOKUP "_COPY" ; LOOKUP "10.100.20.42"; LOOKUP "_FH" ; OPEN "0x12345" ; GETFH }

the source server checks if it has authorized copies from 10.100.20.42 
for 0x12345, and if so, GETFH returns 0x12345.
 
> And now I'm popping back from the identification problem to the authorization
> one...
> 
> What I am getting to is that before you can add RPCSEC_GSSv3 as an 
> optional way to secure the inter-server copy, you must first define 
> how it will work without it.

To be clear, we propose that RPCSEC_GSSv3 MUST be implemented by the 
client, source server, and destination server. Given that the ONC RPC 
working group was prevented from advancing along its work until 
RFC2203 was published, we don't believe IESG would approve an NFSv4.x 
spec that did not mandate this type of protection in conforming 
implementations. Nonetheless, mandatory to implement does not mean 
mandatory to use. Customers are free to continue with the ill-advised 
deployment of NFS without RPCSEC_GSS security.

Also, keep in mind that AUTH_SYS in the NFSv4.1 and NFSv4.0 specs is 
an OPTIONAL, not REQUIRED authentication flavor.

   "However, AUTH_SYS is merely an OPTIONAL security flavor in
   NFSv4.1, and so interoperability via AUTH_SYS is not assured."

We are not proposing than a future minor version of NFSv4 change 
AUTH_SYS from OPTIONAL to RECOMMENDED or REQUIRED.

> That implies the "trust model" you allude to in A2. Which I think 
> goes against this on page 19:
> 

No, we're not contradicting the text above. We do not mandate a trust 
relationship (which is what the text is about), but we do allow for 
it. In a configuration without RPCSEC_GSSv3 (e.g. an AUTH_SYS 
configuration), some amount of trust is required between the 
participants. This is true with or without the copy operations.

---

From: Tom Haynes <[email-masked]>
Date: Tue, 07 Apr 2009 17:18:47 -0500

> Let me elaborate using your picture from earlier:

Ahh, the source and the client would not know 10.100.20.42. (and now I 
think that this
is a typo and you didn't change the 2nd octet).

But I'll accept that the following pair of addresses would cover your 
example:

nfs://10.100.10.43//_COPY/10.10.20.42/_FH/0x12345
nfs://10.10.20.43//_COPY/10.10.20.42/_FH/0x12345

> for 0x12345, and if so, GETFH returns 0x12345.

While I don't disagree with this example (and I think you should provide 
it in the draft), you
have left open that the source may return as the list of addresses:

(nfs://10.100.10.43//_COPY/10.100.20.42/_FH/0x1234, 
nfs://10.10.20.43//_COPY/10.10.20.42/_FH/0x12345, 10.100.10.43, 
10.10.20.43, foo-fe.corp, foo-be.corp)

If the destination uses the first two addresses, then it is basically 
using a magic cookie to identify
itself and is also using a handshake to kick things off. I.e., by 
looking up "_COPY"
we are stating that we are a s2s COPY and not a client initiated cp.

If the destination uses the third address (and fifth), then the source 
has no way of knowing that
this is the requested COPY.

If the destination uses the fourth address (and sixth), then the source 
can assume that this is the
requested COPY. But if the destination already has read permissions on 
the file, the
source can not differentiate this from a normal NFSv4 file read.

I happen to think that restricting the NFSv4.x s2s copies to having to 
parse through
these URLs is an acceptable alternative to having the client first get 
the list of
possible methods it will connect through.

It removes all of my concerns about COPY_REVOKE (but not including the user
interaction verbage).

---

From: James Lentini <[email-masked]>
Date: Wed, 8 Apr 2009 08:52:11 -0400 (EDT)

On Tue, 7 Apr 2009, Tom Haynes wrote:

> > > understanding what I am asking here.
> > >     
> > 
> > Note that in an environment without cryptographic authentication, it will be
> > impossible to securely validate an identity. If the goal is to guard against
> > a malicious adversary, then real security should be used.
> > 
> >   
> > > Consider two different destination servers copying the same FH and neither
> > > are using the location that the client provided in the COPY_NOTIFY.
> > > 
> > > > to COPY_NOTICE, and
> > > that out to every client.
> > >     
> > 
> > The source server may reply to the client with a list of target addresses,
> > names, and/or URLs and assign them to the unique triple: <source fh, user
> > ID, destination address Y>. If the destination uses one of those target
> > netlocs (these are netlocs on the source server, not the destination), the
> > source server has now identified the the destination regardless what address
> > the destination connects from.
> > 
> Ahh, the source and the client would not know 10.100.20.42. (and now 
> I think that this is a typo and you didn't change the 2nd octet).

Yes, sorry about that.
 
> But I'll accept that the following pair of addresses would cover your example:
> 
> > the source server checks if it has authorized copies from 10.100.20.42 for
> > 0x12345, and if so, GETFH returns 0x12345.
> >    
> 
> While I don't disagree with this example (and I think you should 
> provide it in the draft), you have left open that the source may 
> return as the list of addresses:
> 
> (nfs://10.100.10.43//_COPY/10.100.20.42/_FH/0x1234, nfs://10.10.20.43//_COPY/10.10.20.42/_FH/0x12345, 10.100.10.43, 10.10.20.43, foo-fe.corp, foo-be.corp)
> 
> If the destination uses the first two addresses, then it is 
> basically using a magic cookie to identify itself and is also using 
> a handshake to kick things off. I.e., by looking up "_COPY" we are 
> stating that we are a s2s COPY and not a client initiated cp.
> 
> If the destination uses the third address (and fifth), then the 
> source has no way of knowing that this is the requested COPY.
> 
> If the destination uses the fourth address (and sixth), then the 
> source can assume that this is the requested COPY. But if the 
> destination already has read permissions on the file, the source can 
> not differentiate this from a normal NFSv4 file read.
> 
> I happen to think that restricting the NFSv4.x s2s copies to having 
> to parse through these URLs is an acceptable alternative to having 
> the client first get the list of possible methods it will connect 
> through.
> 
> It removes all of my concerns about COPY_REVOKE (but not including 
> the user interaction verbage).

Great.

We will work on an update to the draft that describes this and 
corrects the typos that have been pointed out. I hope to have it ready 
by the end of the week.