Subject: [nfsv4] pNFS MDS and DS sharing clientid

From: Rick Macklem <[email-masked]>
Date: Thu, 29 Dec 2011 20:25:00 -0500 (EST)

I've read Sec. 13.1 of rfc5661 a couple of times and am still
confused (always:-), so I thought someone could help clarify
the case where the server replies with both:
  EXCHGID4_FLAG_USE_PNFS_MDS | EXCHGID4_FLAG_USE_PNFS_DS

If I understood it correctly (quite possibly not), this means
that the server can handle both MDS and DS RPCs using the
same ClientID replied by the EXCHANGEID Op.

If the above is correct, for the File layout case, how does the
client know that the server in struct nfsv4_1_file_layout_ds_addr4
reply for a GETDEVICEINFO is the same as the above MDS+DS?
(Is it simply "same IP address"?)

Alternately, if I always do a separate EXCHANGEID, CREATE_SESSION on
a different TCP connection to a DS in "struct nfsv4_1_file_layout_ds_addr4"
(even if the MDS with EXCHGID4_FLAG_USE_PNFS_DS as above), will the
server allow that and reply with a separate ClientID and SessionID
that can be used for the connection to the DS?

Thanks in advance for any help, rick

---

From: <[email-masked]>
Date: Fri, 30 Dec 2011 09:08:01 -0500

> same ClientID replied by the EXCHANGEID Op.

I agree.

> reply for a GETDEVICEINFO is the same as the above MDS+DS?

Why does he care about that question specifically?

> (Is it simply "same IP address"?)

If it isn't, then it is  "same IP address or different IP address 
connected to the same server (as shown by server owner)", but I don't
know which of those two is right.

The client needs to take the layout and establish a connection with the DS.
So the first thing he does is to see if he has a session with that
address that has the USE_PNFS_DS bit set, and if he does, he uses it.
(and he might also allow one to a different IP that is for the same
Server).  I don't think that the fact that that server also set 
USE_PNFS_MDS changes anything here.

> Alternately, if I always do a separate EXCHANGE_ID, CREATE_SESSION on
> a different TCP connection to a DS in "struct NFSv4.1_file_layout_ds_addr4"

I expect that the server will give you back the same clientid that you got
the first time and you can use it do DS stuff (and MDFS stuff)

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Rick Macklem
Sent: Thursday, December 29, 2011 8:25 PM
To: nfsv4
Subject: [nfsv4] pNFS MDS and DS sharing clientid

Thanks in advance for any help, rick
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Rick Macklem <[email-masked]>
Date: Fri, 30 Dec 2011 12:03:00 -0500 (EST)

> know which of those two is right.
Ok, my question comes down to "How does the client know it's the
same server", see below. (For the case where the MDS doesn't set
EXCHGID_FLAG_USE_PNFS_DS it is not the same server.)

> the first time and you can use it do DS stuff (and MDFS stuff)
Ok, here's where I was stuck:
- client connects to MDS and does EXCHANGEID, etc (Getting a reply with
  both EXCHGID4_FLAG_USE_PNFS_MDS and EXCHGID4_FLAG_USE_PNFS_DS set.)
- at some point, the client needs to do I/O, so it does a
  LAYOUTGET followed by a GETDEVICEINFO for the deviceid4

Now, the client has a netaddr for a DS server it needs to talk to and
it already has an established clientid/session for the MDS.
--> This was where I couldn't decide if/when I could use the already
    established TCP connection with clientid/session.

>From your comment above, is the correct thing for the client to do...
- create a TCP connection to the netaddr4 in the reply from GETDEVICEINFO
- do an EXCHANGEID4 on that TCP connection with the same
   eia_clientowner it used for EXCHANGEID on the MDS above
--> Then use the reply from this EXCHANGEID to determine "same server"
    as the MDS above.
    - If same
         - just close this TCP connection and use the established one
           for the MDS above
      else (not same)
         - do a CREATE_SESSION and use this established session for
           the DS I/O operations

Does this variant sound right? (When I first posted, I had assumed the
"same server" decision needed to be made before doing a new TCP connection
and EXCHANGEID4.)

Thanks for the help, rick

---

From: <[email-masked]>
Date: Fri, 30 Dec 2011 12:24:13 -0500

> it already has an established clientid/session for the MDS.

I believe that it is best to treat this exactly the same as any other situation 
in which we have a session for a clientid for which USE_DS was returned.  
Whether USE_MDS was also returned doesn't/shouldn't affect the issue.

>     established TCP connection with clientid/session.

I say that I the original EXCHANGED_ID got back USE_DS, and that session
is still active, then you are done, and should use that session

If you did a new EXCHANGEID and the server was not the same, then there
is a problem with the server, or it just happened to reboot in which case 
you would find that out when using the existing session.

---

From: Rick Macklem <[email-masked]>
Date: Fri, 30 Dec 2011 12:41:21 -0500 (EST)

> Whether USE_MDS was also returned doesn't/shouldn't affect the issue.
Ok, I see what you are saying. I was just worrying about the 1st time
I needed to talk to a DS, but I can see that the same would apply to
subsequent cases of needing a clientid/session for an address in
nfsv4_1_file_layout_ds_addr4.

> is still active, then you are done, and should use that session
But if I don't do what I suggested below, "How do I know whether or
not the address in nfsv4_1_file_layout_ds_addr4.nflda_multipath_ds_list<N>
is the "same server" as the MDS?

Without a new EXCHANGEID, all I can see the client can do is compare
the netaddr4 host IP address with the one provided at mount time for
the NFSv4.1 (MDS) server. And do I compare all of the ones in
multipath_list4<>, looking for a match in this way?

It seems that a bunch of stuff was put in the EXCHANGEID reply,
like server_scope, so that "same/not same server" could be decided,
but only EXCHANGEID gets that information.

Still confused, but thanks for the help, rick

> > I expect that the server will give you back the same clientid that
> > you
> > got
> > From: [email-masked] [mailto:[email-masked]] On
> > Behalf
> > Of Rick Macklem