Subject: [nfsv4] Session-based fix for callback races

From: "Talpey, Thomas" <[email-masked]>
Date: Mon, 13 Feb 2006 17:04:23 -0500

I have been on the hook to propose some text for the sessions-based
solution to the callback problem as part of the minor version discussions.
This is a proposal for a mechanism to fix the issue of the client's response
to a server callback which arrives before the server's original reply.

The section numbers are approximately where the text would insert into
the minor version doc, draft-ietf-nfsv4-minorversion1-01.txt. You may
want to scan the lead-in section for context.

Some comments are enclosed below in [...]

Feel free to comment, this section is just a first cut at the draft's text.
By the way, the presence of both the clientid and sessionid fields in
the CB_SEQUENCE text below is outdated, and are to be cleaned up
(the clientid is not needed since it is subsumed into the sessionid).

Tom.

-----

3.10.2 Slot Identifiers and Server Duplicate Request Cache  . . .

<please see existing section>

3.10.3 Resolving server callback races with sessions [new]

It is possible for server callbacks to arrive at the client before the
reply from related forward channel operations. For example, a client may
have been granted a delegation to a file it has opened, but the reply
to the OPEN (informing the client of the granting of the delegation) may
be delayed in the network. If a conflicting operation arrives at the
server, it will recall the delegation using the callback channel, which
may be on a different TCP connection, perhaps even a different network.
If the callback request arrives before the related reply, the client may
reply to the server with an error.

The presence of a session between client and server can be used to
alleviate this issue. When a session is in place, each client request
is uniquely identified by its {slotid, sequenceid} pair. By the rules
under which slot entries (duplicate request cache entries) are retired,
the server has knowledge whether the client has "seen" each of the server's
replies. The server can therefore provide sufficient information to the
client to allow it to disambiguate between an erroneous callback and a
race condition.

To implement this, the CB_SEQUENCE operation which begins each server
callback may optionally carry a related {slotid, sequenceid} identifier.
If the client finds this identifier to be currently outstanding (the
server's reply has not been seen by the client), it can determine that
the callback has raced the reply, and act accordingly.

The client must not simply wait forever for the expected server reply
to arrive any of the session's operations channels, because it is possible
that they will be delayed indefinitely. However, it should endeavor to wait
for a period of time, and if the time expires it can provide a more
meaningful error such as NFS4ERR_DELAY.

[We need to consider the clients' options here, and describe them...
 NFS4ERR_DELAY has been discussed as a legal reply to CB_RECALL? ]

There are other scenarios under which callbacks may race replies, among
them pnfs layout recalls, [fill in the blanks w/others, etc...]

Therefore, for each client operation which might result in some sort of
server callback, the server should "remember" the {slotid, sequenceid} pair
of the client request until the slotid retirement rules allow the server
to determine that the client has, in fact, seen the server's reply.
During this time, any recalls of the associated object should carry these
identifiers, for the benefit of the client. After this time, it is not
necessary for the server to provide this information in related callbacks,
since it is certain that a race condition can no longer occur.

-----

14.10.  Callback operation: CB_SEQUENCE - Supply callback channel
sequencing and control

     SYNOPSIS

          control -> control

     ARGUMENT

          typedef uint32_t sequenceid4;
          typedef uint32_t slotid4;

          struct CB_SEQUENCE4args {
               clientid4 clientid;
               sessionid4     sessionid;
               sequenceid4    sequenceid;
               slotid4        slotid;
               slotid4        maxslot;
>>             sequenceid4    referring_sequenceid;
>>             slotid4        referring_slotid;
          };

     RESULT

          struct CB_SEQUENCE4resok {
               clientid4 clientid;
               sessionid4     sessionid;
               sequenceid4    sequenceid;
               slotid4        slotid;
               slotid4        maxslot;
               slotid4        target_maxslot;
          };

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4