Subject: [nfsv4] parallel LAYOUTGETs using open stateid

From: Fred Isaman <[email-masked]>
Date: Thu, 7 Oct 2010 09:21:18 -0400

This question came up as I was going through our layout stateid code.

Can the client send multiple LAYOUTGETs in parallel before getting a
layoutstateid by using an open stateid?
If so, what is the server response?  The spec is not clear, but I
would assume it would do one of:
a) returning an error (what error?) on all but the first LAYOUTGET it sees
b) matching the openstateid to the layoutstateid it has already given
out and sending a reply with that stateid and a bumped seqid

Fred

---

From: Benny Halevy <[email-masked]>
Date: Thu, 7 Oct 2010 09:50:35 -0400

> out and sending a reply with that stateid and a bumped seqid

I'm in favour of (b).
Are there any problems you see in that approach?

Benny

> Fred
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Benny Halevy <[email-masked]>
Date: Thu, 07 Oct 2010 10:18:47 -0400

>> out and sending a reply with that stateid and a bumped seqid

Just to clarify, what matters in this case is the returned logr_stateid
and seqid while the sent loga_stateid does not matter much.
If there's a pending layout recall on the server which has not been
completed by a matching LAYOUTRETURN the server can safely assume
that the layoutget was sent before the client received the CB_LAYOUTRECALL
and return NFS4ERR_RECALLCONFLICT.

Benny

---

From: Fred Isaman <[email-masked]>
Date: Thu, 7 Oct 2010 10:20:32 -0400

> Are there any problems you see in that approach?

The wrinkle it adds is that, in the normal case, after the CB_RECALL
is resolved, any stale LAYOUTGETs (sent by the client before it
received the CB_RECALL) that the server receives will either be
processed normally with a bumped seqid or an error (if the server
invalidated the "other" part of the stateid.)  In the openstateid
case, the server loses the option of returning an error, instead it
wuold create a new "other" with seqid=1.  That said, I don't see that
this causes any problems apart from extra bookkeeping.

Fred

---

From: Benny Halevy <[email-masked]>
Date: Thu, 07 Oct 2010 10:36:57 -0400

> processed normally with a bumped seqid or an error (if the server

But the client MUST not process the layout recall before receiving
replies for the outstanding layoutgets. (see section 12.5.5.2.1.
Layout Recall and Return Sequencing)

Benny

---

From: Fred Isaman <[email-masked]>
Date: Thu, 7 Oct 2010 11:28:17 -0400

> Layout Recall and Return Sequencing)

No, it must wait for the outstanding layoutgets that the server
processed before sending the CB_RECALL.  It is possible (at least by
the way I read the spec), to send two LAYOUTGETs, one of which gets
delayed and doesn't hit the server until after the CB_RECALL is
resolved, and then have the server process that LAYOUTGET normally.

Fred

---

From: "William A. (Andy) Adamson" <[email-masked]>
Date: Thu, 7 Oct 2010 11:32:27 -0400

> Layout Recall and Return Sequencing)

Hmmm. But 12.5.5.2.1 is referring to outstanding layoutget's with the
same stateid as the cb_layoutrecall. If the server creates "new" ones
as described below by Fred, the client doesn't need to wait. Of
course, the client has no way of knowing until it get's the response
:)

-->Andy

---

From: Benny Halevy <[email-masked]>
Date: Thu, 07 Oct 2010 11:47:41 -0400

> course, the client has no way of knowing until it get's the response

The spec describes that situation as:
   the "seqid" is not exactly one higher than what
   the client currently has recorded.

With the initial open/deleg/lock stated, the "other" part
of the stateid is different - therefore one can assume
the seqid if not exactly one higher than ... so I think that the
client MUST wait as described in the is section.

Though, I agree the can be improved and clarified.
In particular:
   The client MUST wait before
   processing such a CB_LAYOUTRECALL until it processes all replies for
   outstanding LAYOUTGET and LAYOUTRETURN operations for the
   corresponding file with seqid less than the seqid given by
   CB_LAYOUTRECALL (lor_stateid; see Section 20.3.)

should explicitly cover outstanding LAYOUTGETs with non-layout stateid.

Benny