Subject: [nfsv4] nfsv4 over tcp drc

From: [email-masked]
Date: Sun, 5 Mar 2006 17:36:03 -0500 (EST)

fyi, related to my talk at cthon last week, I remembered that my main
reason for leaving client IP address out of the cache key was concerns
w.r.t. DHCP. If the client is network partitioned from the server, it
could also be network partitioned from the DHCPd. It seems to me it
could get a different IP address when the network partition heals.

rick

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Talpey, Thomas" <[email-masked]>
Date: Mon, 06 Mar 2006 08:23:16 -0500

Yes, and that's a real problem for the DRC. But the problem is that
the XID is a lousy uniqueness token. It's only unique in the scope
of a single RPC stream. Linux, for example, takes this literally and
cycles xid's on a per-mount basis. So two mounts from the *same
client* can theoretically collide XIDs if you don't take the address/port
into consideration. Apart from all the other obvious collisions from
N clients.

My concerns about the weakness of DRC implementations using
transport-specific context are, however, well-documented in the
Sessions draft, which now the minorversion one, so I won't babble
on. Still, I laud your attempt at making do with the old protocol! :-)

Tom.

---

From: Eric Werme USG <[email-masked]>
Date: Mon, 06 Mar 2006 13:37:06 -0500

Thomas.Talpey wrote:

   Linux, for example, takes this literally and
   cycles xid's on a per-mount basis. So two mounts from the *same
   client* can theoretically collide XIDs if you don't take the address/port
   into consideration.

Tru64 Unix tracks address and port in the DRC for both UDP and TCP.
I added the port (in 1995!) because some customer bought a user level
application that recycled XIDs way too quickly.  The retransmitted from
the same port as the first request so adding the port number improved
the uniqueness test substantially.

My NFS over TCP implementation can break a connection before the client
receives a reply.  Unfortunately, the client will create a new
connection but it will use a new port number.  I timeout drc entries
after 120 seconds, so if things get screwed up enough to break a
connection, by the time a new one can be established, there's a good
chance the cache entry has been discarded.

   >could get a different IP address when the network partition heals.

A group here is trying to use Linux clients (and servers, I believe) with
the clients hooked into multiple NAT boxes.  If one NAT box fails, the
other will take over.  It mostly works, but it completely falls apart
with file locking due to the IP address of the lock holder changing.
"It appears that if I Iock a file from a client behind a NAT server and
the NAT server fails, I can't release the lock if the client reconnects
to the NFS via a different NAT server (hence a different IP address)."

So add locking to the morass!

BTW, Tru64 clients select an initial XID based on Unix time
(time.tv_sec << 12).  That was to deal with false DRC hits when
we started at zero and someone installed some software over NFS
that needed a couple reboots.  Reboot, send a write with XID 13,
reboot, send a different write with XID 13, server says "I saw that!"
and doesn't bother with the write.

Boot a bunch of Tru64 clients at the same time, have them talk to the
same server and having the IP address in the drc cache becomes rather
important.  Of course, it gets less and less likely that you'll have
to worry about that case.  :-(

That was before we timed out drc entries.  There are more notes on
our XID selection at http://www.connectathon.org/talks96/werme1.html .
It looks like we also had problems with OS/2 NFS reusing XIDs.

	-Ric Werme

---

From: "Talpey, Thomas" <[email-masked]>
Date: Mon, 06 Mar 2006 14:39:14 -0500

The more things change, the more they stay the same, eh?
As for NLM locking, DRC issues are the least of its worry! Well,
"among its worries" anyway! :-)

Your talk (that you mention below) is referenced in the sessions
text of the new minor version, btw.

Tom.

At 01:37 PM 3/6/2006, Eric Werme USG wrote:
>connection but it will use a new port number.
>...
>...
>...
>There are more notes on