Subject: [nfsv4] detailed NOTIFY_DEVICEID4_CHANGE semantics?

From: Christoph Hellwig <[email-masked]>
Date: Mon, 1 Sep 2014 04:03:57 +0200

RFC5661 is very sparse on NOTIFY_DEVICEID4_CHANGE semantics, and the
other layout RFCs don't mention it at all.  I'm particularly worried about
the very imprecise language that talks about outstanding I/O, but does
not mention outstanding layouts, which are the objects that refer to
deviceids in pNFS:

   NOTIFY_DEVICEID4_CHANGE
      A previously provided device-ID-to-device-address mapping has
      changed and the client uses GETDEVICEINFO to obtain the updated
      mapping.  The notification is encoded in a value of data type
      notify_deviceid_change4.  This data type also contains a boolean
      field, ndc_immediate, which if TRUE indicates that the change will
      be enforced immediately, and so the client might not be able to
      complete any pending I/O to the device ID.  If ndc_immediate is
      FALSE, then for an indefinite time, the client can complete
      pending I/O.  After pending I/O is complete, the client SHOULD get
      the new device-ID-to-device-address mappings before sending new
      I/O requests to the storage devices addressed by the device ID.

What is a pending I/O in this case?  As far as the MDS is concerned I/O
operations aren't visible in pNFS, we just have outstanding layouts
that can be commited and/or returned.

For my block layout client and server implementation I'm interpreting the
ndc_immediate = FALSE case in the following way:

 The client may finish any I/O using outstanding layouts that reference the
 previously provided device-ID-to-device-address mapping, and SHOULD use
 GETDEVICEINFO to obtain the updated mapping before requesting new layouts
 for the previously provided device-ID-to-device-address mapping.

Does this sound like reasonable language to put into RFC5661bis?

What is the use case for ndc_immediate = TRUE, and how is a client supposed
to handle it?

---

From: Benny Halevy <[email-masked]>
Date: Mon, 01 Sep 2014 09:51:56 +0300

> Does this sound like reasonable language to put into RFC5661bis?

The new device info applies to the current layout so I see no reason to wait
with a GETDEVICEINFO until the client gets new layouts as the server is retiring
the old mapping in the background.  To prevent needless I/O errors caused by using
a stale mapping I'd suggest the following instead:

  The client may finish any outstanding I/Os that reference the previously provided
  device-ID-to-device-address mapping and SHOULD use GETDEVICEINFO to obtain the
  updated mapping before issuing new I/Os that reference the device ID via either
  outstanding layouts or layouts acquired after receiving the deviceid change
  notification.

Benny

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>