Subject: [nfsv4] On the v4 referral mechanism

From: Nicolas Williams <[email-masked]>
Date: Mon, 22 Dec 2003 17:20:18 -0600

There's been a lot of confusion lately over just how referrals happen.

Here's my take.

Take a client, C traversing a server-side path of /a/b/c on server S1
where the last component does not exist on the server but will elicit a
referral to some other server, S2.  The client will discover the
referral as follows:

 - Case 1: client generally uses LOOKUP and GETATTR in the same compound

    - C ->S1 {PUTROOTFH, LOOKUP("a"), GETATTR(fs_locations4), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, "S1:/a", FH_a}

    - C ->S1 {PUTFH(FH_a), LOOKUP("b"), GETATTR(fs_locations4), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, "S1:/a", FH_b}

    - C ->S1 {PUTFH(FH_b), LOOKUP("c"), GETATTR(fs_locations4), GETFH,
              READDIR(...)}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x/y (rootdir: /x)", NFS4ERR_MOVED,
              NFS4ERR_MOVED}

    - C ->S2 {PUTROOTFH, LOOKUP("x"), GETATTR(fs_locations4), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x", FH_x}

    - C ->S2 {PUTFH(FH_x), LOOKUP("y"), GETATTR(fs_locations4), GETFH,
              READDIR(...)}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x", FH_y, <READDIR4resok>}

 - Case 2: client generally does not GETATTR in the same compound as
           LOOKUP

    - C ->S1 {PUTROOTFH, LOOKUP("a"), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, FH_a}

    - C ->S1 {PUTFH(FH_a), LOOKUP("b"), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, FH_b}

    - C ->S1 {PUTFH(FH_b), LOOKUP("c"), GETFH, READDIR(...)}
      S1->C  {NFS4_OK, NFS4_OK, NFS4ERR_MOVED, NFS4ERR_MOVED}

    - C ->S1 {PUTFH(FH_b), LOOKUP("c"), GETATTR(fs_locations4)}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x/y (rootdir: /x)"}

    - C ->S2 {PUTROOTFH, LOOKUP("x"), GETATTR(fs_locations4), GETFH}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x", FH_x}

    - C ->S2 {PUTFH(FH_x), LOOKUP("y"), GETATTR(fs_locations4), GETFH,
              READDIR(...)}
      S1->C  {NFS4_OK, NFS4_OK, "S2:/x", FH_y, <READDIR4resok>}

As you can see, for the common case including a GETATTR(fs_locations4)
with every LOOKUP is wasteful in the common case (no referral).  But not
including a GETATTR(fs_locations4) with every LOOKUP means doing an
extra round-trip.

One possible way to optimize case 1, if we should optimize it at all,
could be to say that the server replies to {PUT*FH(...), LOOKUP(...),
GETATTR(fs_locations4), ... } compounds with no fs_locations4 in the
GETATTR reply unless there is a referral, but otherwise always includes
an fs_locations4 in replies to GETATTR(fs_locations4) in other compound
combinations (e.g., {PUT*FH(...), GETATTR(fs_locations4)}).

Cheers,

Nico
-- 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 30 Dec 2003 12:08:28 -0800

This mostly matches my understanding.  The only change in the
scenarios I would make is that the first error terminates the 
COMPOUND so GETFH-READDIR only results in a single NFS4ERR_MOVED.

As to the optimization issue of whether to include the GETATTR,
I would not do this unless I thought a referral has a significant
probability.  So I would never do it unless I was in the root
(i.e. pseudo-fs) for a given server.  You can have a referral
within a normal (i.e. non-pseudo) fs, but the probability per
lookup is liable to be orders of magnitude less.  So in that 
case, the extra turnaround would not be that big a deal.  Also
note, that a referral in that case, is a subcase of the crossing-
an-fs-boundary case.  So my guess is that as you descend the
directory tree you will be looking for fsid changes and that when
you note that you will be fetching a number of per-fs attributes 
in a second request (if you keep it just one) when you set of that
subsidiary mount point.  So even in the non-referral case, when 
you cross an fs boundary, I wouldn't expect that all to happen
without at least one extra request.

I think is legal to give a very minimal fs_locations when there is
no replica or referral or migration of any kind.  I think the client
has to be prepared to take this as a steady-as-she-goes indication.
As far as making the value returned context-specific, I'm a little
wary.  If S1's pseudo-fs had a replica at S1a's pseudo's, my thinking
would be that it would be best to return both of these when GETATTR 
requested it, whatever the context.  In any case, that would be a 
pre-computed per-fs array that could be delivered to requestors without 
a lot of overhead.

-----Original Message-----
From: Nicolas Williams [mailto:[email-masked]]
Sent: Monday, December 22, 2003 6:20 PM
To: [email-masked]
Cc: Noveck, Dave; Carl Burnett
Subject: On the v4 referral mechanism

Here's my take.

Cheers,

Nico
--