Subject: Re: Conference Call Proposed (was RE: [nfsv4] Changes to GETDEVICELIST in NFSv41 I-D 15 and 18)

From: Mike Eisler <[email-masked]>
Date: Thu, 24 Jan 2008 14:14:26 -0800 (PST)

--- Mike Eisler <[email-masked]> wrote:

> Folks who plan to call in, or have a proxy call in should respond
> *directly* to me. On Thursday, January 24, I will determine if there
> will
> be sufficient attendance, and if so, provide dial in details to
> all participants.

We have quorum, and representation from all known and presumed
pNFS implementors but one. Therefore the call will happen on January
25, NOON pacific time as planned. All respondents have been sent
dial in details. If anyone else wants to participate, or if
you responded and did not receive details, please email: 

  [email-masked]

and make sure the Subject: like of your email is:

  Re: Conference Call Proposed (was RE: [nfsv4] Changes to
GETDEVICELIST in NFSv41 I-D 15 and 18)

Thanks.

   -mre

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Mike Eisler <[email-masked]>
Date: Fri, 25 Jan 2008 13:35:22 -0800 (PST)

--- Mike Eisler <[email-masked]> wrote:

> 25, NOON pacific time as planned. 

Notes from Third pNFS formal review – 2008-01-25

Copyright © The IETF Trust (2008).

Attendees:

Jason Glasgow
Lisa Week
Sam Falkner
Marc Eshel
Robert Gordon
Rick Mesta
Spencer Shepler
Benny Halevy
Ricardo Labiaga
Dave Noveck
Mike Eisler
Andy Adamson
Mario Wurzl
Dean Hildebrand
Trond Myklebust
Brent Welch

Mike: took roll call, and then informed everyone that this was an
official IETF meeting and everything said was considered an IETF
contribution. 

Jason: Getdevicelist is to be used which devices can be used for a file
system. In draft-18, there is no longer an fsid argument. Asking for
all devices without an fsid is useless. You want notifications for all
file systems you are interested in, but you want to make sure that you
can cancel the notification when you are no longer interested in it.
Complexity arises in notifications. If you want notifications you have
to get them for all filesystems. One way to solve this would be to have
an fsid attribute, but if so, why have GETDEVICELIST.

Dave: Do we have general agreement that getdevicelist for an fsid is
needed?

Mike: isn’t it simpler, just have an indication whether an fsid
supports pNFS

Trond: why is it simpler to do GDL on an entire system vs. per-fsid?  
Why couldn't a person configure things such that a server has only one
file system?  Sure, if that configuration exists, we don't have the
problem, but it is not likely that configurations will always be like
that.

Marc: it would be simpler if there was an association between deviceIDs
and fsids.

Mario: Easily associate a FS to a set of devices.  Be able to see if
all devices for a file system are available to the client or not.

Jason: Answering Mike’s question: no, Block pNFS client needs to be
able to query the server to figure out which disks are required to do
pNFS on the file system of interest.

Someone said: Can we have notifications be for GETDEVICEINFO (GDI)
only? 

Jason:  It is not optimal, but doable.  The problem is that in order to
get notifications you have to re-get information that the client
already has.

Andy: Objects and Files don't really need GDL.

Andy: Simplest thing to just eliminate ADD.  No notifications on GDL.

Jason: that would be workable, but then he’d have to issue a
GETDEVICEINFO to set notifications.

Brent: the issue is how the client detects that the info is stale.

Andy: I don’t think we care about notifications for a device I am not
using.

Brent: what about a device that comes into existence later?

Andy: you find out about the new device when you get a layout.

Andy: eliminate _ADD, eliminate notifications from GETDEVICELIST

Noveck: complexity comes around if setting notifications on
GETDEVICELIST per fsid, how does one deal with different notification
masks.

Benny: I sent a proposal that used a SETDEVICEID.

Jason: it was a similar idea to Dave’s and as complex.

Mike: in hindsight I would have had GETDEVICELIST return just device
IDs.

Spencer: that would be my preference.

Spencer: In Vancouver, my impression not too many people were
interested in GETDEVICELIST. Reducing scope of function is the safer
approach.

Dave: Good point, notifications are complex, eliminating them from one
operation simplifies.

Summary of Proposal:

1. GDL just returns *just* an array of DevicesIDs
2. GDL is per-fsid.
3. There are no notifications on GDL.
4. GDI is the only way to get device addresses and notifications of
CHANGE and DELETE of/to existing deviceIDs.  ADD is eliminated from the
protocol.
5. You can also use GDI to turn off notifications.

Re-statement of items that will *NOT* be changing:
1. Scope of DeviceID:  A DeviceID is global per ClientID/Layout Type
pair. (GDL should continue to take the Layout Type and it gets the
ClientID from the Session.)
2. GDL remains optional to implement in the scope of pNFS.

Consensus was that this was a good solution.

Mike will send out the change-differences to the WG alias today.  Then
will give everyone a chance to review it.  Deadline for comments COB
Monday EST.  
Spencer will submit draft-19 after that.

---

From: Mike Eisler <[email-masked]>
Date: Fri, 25 Jan 2008 15:36:38 -0800 (PST)

> Spencer will submit draft-19 after that.

Attached.

Index: Makefile
===================================================================
RCS file: /cvs/stdk/NFSv41/Makefile,v
retrieving revision 1.107
diff -u -w -b -r1.107 Makefile
--- Makefile	31 Dec 2007 22:51:34 -0000	1.107
+++ Makefile	25 Jan 2008 23:13:50 -0000
@@ -121,7 +121,6 @@
 		dotx.d/type_ssv_seal_plain_tkn4.x \
 		dotx.d/type_ssv_seal_cipher_tkn4.x \
 		dotx.d/type_layoutreturn4.x \
-		dotx.d/type_devlist_item4.x \
 		dotx.d/type_client_owner4.x \
 		dotx.d/type_server_owner4.x \
 		dotx.d/type_device_addr4.x \
@@ -186,7 +185,6 @@
 		autogen/type_ssv_seal_plain_tkn4.xml \
 		autogen/type_ssv_seal_cipher_tkn4.xml \
 		autogen/type_layoutreturn4.xml \
-		autogen/type_devlist_item4.xml \
 		autogen/type_client_owner4.xml \
 		autogen/type_server_owner4.xml \
 		autogen/type_device_addr4.xml \
Index: nfsv41_middle_datatypes.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_datatypes.xml,v
retrieving revision 1.42
diff -u -w -b -r1.42 nfsv41_middle_datatypes.xml
--- nfsv41_middle_datatypes.xml	22 Dec 2007 18:56:39 -0000	1.42
+++ nfsv41_middle_datatypes.xml	25 Jan 2008 23:13:50 -0000
@@ -342,17 +342,6 @@
       </t>
     </section>
 
-    <section toc="exclude" anchor="devlist_item4"
-	     title="devlist_item4">
-	 <?rfc include='autogen/type_devlist_item4.xml'?>
-      <t>
-        An array of these values is returned by the GETDEVICELIST
-	operation.  They define the set of devices associated with a
-	file system for the layout type specified in the
-	GETDEVICELIST4args.
-      </t>
-    </section>
-
     <section toc="exclude" anchor="layout_content4" title="layout_content4">
 	 <?rfc include='autogen/type_layout_content4.xml'?>
       <t>
Index: nfsv41_middle_errors.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_errors.xml,v
retrieving revision 1.54
diff -u -w -b -r1.54 nfsv41_middle_errors.xml
--- nfsv41_middle_errors.xml	17 Dec 2007 22:17:39 -0000	1.54
+++ nfsv41_middle_errors.xml	25 Jan 2008 23:13:51 -0000
@@ -1946,7 +1946,6 @@
       NFS4ERR_REP_TOO_BIG_TO_CACHE,
       NFS4ERR_REQ_TOO_BIG,
       NFS4ERR_SERVERFAULT,
-      NFS4ERR_TOOSMALL,
       NFS4ERR_TOO_MANY_OPS,
       NFS4ERR_UNKNOWN_LAYOUTTYPE,
       NFS4ERR_UNSAFE_COMPOUND
Index: nfsv41_middle_op_cb_notify_deviceid.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_cb_notify_deviceid.xml,v
retrieving revision 1.5
diff -u -w -b -r1.5 nfsv41_middle_op_cb_notify_deviceid.xml
--- nfsv41_middle_op_cb_notify_deviceid.xml	22 Dec 2007 18:56:39 -0000	1.5
+++ nfsv41_middle_op_cb_notify_deviceid.xml	25 Jan 2008 23:13:51 -0000
@@ -3,7 +3,7 @@
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14: CB_NOTIFY_DEVICEID - Notify directory changes" >
   <t>
-    Tell the client of directory changes.
+    Tell the client of deviceID changes.
   </t>
 
   <section toc="exclude" anchor="OP_CB_NOTIFY_DEVICEID_ARGUMENT" title="ARGUMENT">
@@ -54,15 +54,6 @@
      The server will send the following notifications:
 
       <list style="hanging">
-	<t hangText="NOTIFY_DEVICEID4_ADD"><vspace />
-	  Adds a device ID to the mappings.  This is
-	  used for a device that has not been previously
-	  returned to the client.  The client uses
-	  GETDEVICEINFO or GETDEVICELIST to obtain the new
-	  mapping. The notification is encoded in a value of
-	  data type notify_deviceid_add_or_delete4.
-
-	</t>
 
 	<t hangText="NOTIFY_DEVICEID4_CHANGE"><vspace />
 	  A previously provided device ID to device
Index: nfsv41_middle_op_getdeviceinfo.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdeviceinfo.xml,v
retrieving revision 1.16
diff -u -w -b -r1.16 nfsv41_middle_op_getdeviceinfo.xml
--- nfsv41_middle_op_getdeviceinfo.xml	17 Dec 2007 22:17:39 -0000	1.16
+++ nfsv41_middle_op_getdeviceinfo.xml	25 Jan 2008 23:13:51 -0000
@@ -37,7 +37,7 @@
       three enumerated values of notify_type4 currently
       apply to GETDEVICEINFO (and GETDEVICELIST,
       see <xref target="OP_GETDEVICELIST" />):
-      NOTIFY_DEVICEID4_ADD NOTIFY_DEVICEID4_CHANGE,
+      NOTIFY_DEVICEID4_CHANGE
       and NOTIFY_DEVICEID4_DELETE (see <xref
       target="OP_CB_NOTIFY_DEVICEID" />).
 
@@ -108,35 +108,10 @@
     </t>
     
     <t>
-       CB_NOTIFY_DEVICEID adds a device ID.  If the client
-       believes it has layouts that refer to the device
-       then the client likely received the
-       device ID in a recent response to a LAYOUTGET,
-       and CB_NOTIFY_DEVICEID is racing with LAYOUTGET
-       as well as GETDEVICEINFO. The client can ignore
-       the CB_NOTIFY_ADD.
-
-       If the client does not have layouts that refer to
-       the added device ID, and GETDEVICEINFO (and not
-       GETDEVICELIST) is racing with CB_NOTIFY_DEVICEID,
-       this begs the question why client is issuing a
-       GETDEVICEINFO on the device ID. A legitimate
-       scenario is that the layout was recalled and
-       returned before the layout could be used. An
-       associated scenario is that the server is also
-       sending a CB_NOTIFY_DEVICEID to delete the device
-       ID, yet another race.  The client should deal with
-       the race by issuing GETDEVICEINFO on the device ID
-       and update (or delete, if GETDEVICEINFO indicates
-       the device ID is not in use) the device ID's device
-       address mappings.
-    </t>
-
-    <t>
        CB_NOTIFY_DEVICEID indicates a device ID's device
        addressing mappings have changed. The client should assume
-       that the results from the in progress GETDEVICEINFO or
-       GETDEVICELIST will be stale for the device ID
+       that the results from the in progress GETDEVICEINFO
+       will be stale for the device ID
        once received, and so it should send a GETDEVICEINFO
        on the device ID.
      
Index: nfsv41_middle_op_getdevicelist.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdevicelist.xml,v
retrieving revision 1.15
diff -u -w -b -r1.15 nfsv41_middle_op_getdevicelist.xml
--- nfsv41_middle_op_getdevicelist.xml	14 Dec 2007 18:17:32 -0000	1.15
+++ nfsv41_middle_op_getdevicelist.xml	25 Jan 2008 23:13:51 -0000
@@ -12,82 +12,35 @@
   <section toc="exclude" anchor="OP_GETDEVICELIST_DESCRIPTION" title="DESCRIPTION">
     <t>
       This operation is used by the client to enumerate all of the
-      device ID to device address mappings in use by the server.  An
+      device IDs a server's file system uses. An
       example of the use of this operation is for device types that
       use SAN devices and in these environments it may be helpful for
       a client to determine device accessibility upon first file system
       access.
     </t>
     <t>
-      The client provides the layout type in gdia_layout_type.  Since
+      The client provides a current filehandle of a file object that
+      belongs to the file system (i.e. all file objects sharing the same
+      fsid as the than of current filehandle), and the layout type
+      in gdia_layout_type.  Since
       this operation may require multiple calls to enumerate all the
       device ID to device address mappings (and is thus
       similar to the <xref target="OP_READDIR">
       READDIR</xref> operation), the client also provides gdia_cookie
       and gdia_cookieverf to specify the current cursor position in the
       mappings.
-      The client provides gdla_maxcount to limit the number of bytes
-      for the result.  This maximum size represents all of the data
-      being returned within the GETDEVICELIST4resok structure and
-      includes the XDR overhead.  The server may return less data.  If
-      the server is unable to return a single entry device list item
-      within the gdla_maxcount limit, the error NFS4ERR_TOOSMALL will
-      be returned.
+      The client provides gdla_maxdevices to limit the number of device IDs
+      for the result.
+      The server MAY return fewer device IDs.
     </t>
     
     <t>
-      The client provides list of notification
-      types, and the server responds with the list it will
-      send in CB_NOTIFY, in the same manner as described in <xref
-      target="OP_GETDEVICEINFO" />.
-    </t>
-    <t>
       The successful response to the operation will contain the
       cookie, gdlr_cookie, and cookie verifier, gdlr_cookieverf, to be
       used on the subsequent GETDEVICELIST.  A gdlr_eof value of TRUE
       signifies that there are no remaining entries in the server's
-      device list.  Each element of gdlr_devinfo_list contains
-      a device ID, and the corresponding device address information.
-      The notifications mask is also
-      returned by the server in gdlr_notification.
-    </t>
-    <t>
-      If NFS4ERR_TOOSMALL is returned, the results also contain
-      gdlr_mincount.  The value of gdlr_mincount represents the
-      minimum size necessary to obtain a single entry device list
-      item.
-    </t>
-  </section>
-  <section toc="exclude" anchor="OP_GETDEVICELIST_IMPLEMENTATION" title="IMPLEMENTATION">
-  <t>
-    The client SHOULD request a notification for additions,
-    changes, deletions of to device ID to device address
-    mappings so that:
-
-    <list style="symbols">
-    <t>
-
-      The server can allow the client gracefully use new
-      mappings, without having pending I/O fail abruptly,
-      or force layouts using the device IDs to be recalled
-      or revoked.
-
-    </t>
-    <t>
-
-      The client can maintain an up to date listing of
-      all device IDs and their device addresses by invoke
-      GETDEVICEINFO on single device IDs versus making
-      periodic calls to GETDEVICELIST.
-
+      device list.  Each element of gdlr_deviceid_list contains
+      a device ID.
     </t>
-    </list>
-
-  </t>
-  <t>
-   See <xref target="OP_GETDEVICEINFO_IMPLEMENTATION"/> for a discussion
-   of races between GETDEVICELIST and CB_NOTIFY_DEVICEID.
-  </t>
-
   </section>
 </section>
Index: dotx.d/Makefile
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/Makefile,v
retrieving revision 1.43
diff -u -w -b -r1.43 Makefile
--- dotx.d/Makefile	10 Jan 2008 16:26:11 -0000	1.43
+++ dotx.d/Makefile	25 Jan 2008 23:13:51 -0000
@@ -166,7 +166,6 @@
 		type_ssv_seal_plain_tkn4.x \
 		type_ssv_seal_cipher_tkn4.x \
 		type_layoutreturn4.x \
-		type_devlist_item4.x \
 		type_client_owner4.x \
 		type_server_owner4.x \
 		type_device_addr4.x \
Index: dotx.d/cb_notify_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/cb_notify_args.x,v
retrieving revision 1.11
diff -u -w -b -r1.11 cb_notify_args.x
--- dotx.d/cb_notify_args.x	20 Dec 2007 20:46:16 -0000	1.11
+++ dotx.d/cb_notify_args.x	25 Jan 2008 23:13:51 -0000
@@ -62,7 +62,7 @@
 typedef opaque notifylist4<>;
 
 struct notify4 {
-	/* composed from notify_type4 */
+	/* composed from notify_type4 or notify_deviceid_type4 */
         bitmap4         notify_mask;
         notifylist4     notify_vals;
 };
Index: dotx.d/cb_notify_deviceid_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/cb_notify_deviceid_args.x,v
retrieving revision 1.2
diff -u -w -b -r1.2 cb_notify_deviceid_args.x
--- dotx.d/cb_notify_deviceid_args.x	10 Dec 2007 19:59:04 -0000	1.2
+++ dotx.d/cb_notify_deviceid_args.x	25 Jan 2008 23:13:51 -0000
@@ -2,15 +2,14 @@
  * Device notification types.
  */
 enum notify_deviceid_type4 {
-	NOTIFY_DEVICEID4_ADD = 0,
 	NOTIFY_DEVICEID4_CHANGE = 1,
 	NOTIFY_DEVICEID4_DELETE = 2
 };
 
-/* For NOTIFY4_DEVICEID4_ADD or NOTIFY4_DEVICEID4_DELETE */
-struct notify_deviceid_add_or_delete4 {
-	layouttype4	ndaod_layouttype;
-	deviceid4	ndaod_deviceid;
+/* For NOTIFY4_DEVICEID4_DELETE */
+struct notify_deviceid_delete4 {
+	layouttype4	ndd_layouttype;
+	deviceid4	ndd_deviceid;
 };
 
 /* For NOTIFY4_DEVICEID4_CHANGE */
Index: dotx.d/get_dir_delegation_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/get_dir_delegation_args.x,v
retrieving revision 1.11
diff -u -w -b -r1.11 get_dir_delegation_args.x
--- dotx.d/get_dir_delegation_args.x	10 Dec 2007 09:27:14 -0000	1.11
+++ dotx.d/get_dir_delegation_args.x	25 Jan 2008 23:13:51 -0000
@@ -1,14 +1,10 @@
-/*
- * Mask of notification types.
- */
-typedef bitmap4 notification_mask4;
 
 typedef nfstime4 attr_notice4;
 
 struct GET_DIR_DELEGATION4args {
 	/* CURRENT_FH: delegated directory */
         bool                    gdda_signal_deleg_avail;
-        notification_mask4  gdda_notification_types;
+        bitmap4		gdda_notification_types;
         attr_notice4            gdda_child_attr_delay;
         attr_notice4            gdda_dir_attr_delay;
         bitmap4                 gdda_child_attributes;
Index: dotx.d/get_dir_delegation_res.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/get_dir_delegation_res.x,v
retrieving revision 1.7
diff -u -w -b -r1.7 get_dir_delegation_res.x
--- dotx.d/get_dir_delegation_res.x	19 Nov 2007 09:56:44 -0000	1.7
+++ dotx.d/get_dir_delegation_res.x	25 Jan 2008 23:13:51 -0000
@@ -3,7 +3,7 @@
 	/* Stateid for get_dir_delegation */
 	stateid4		gddr_stateid;
 	/* Which notifications can the server support */
-	notification_mask4	gddr_notification;
+	bitmap4		gddr_notification;
 	bitmap4			gddr_child_attributes;
 	bitmap4			gddr_dir_attributes;
 };
Index: dotx.d/getdeviceinfo_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/getdeviceinfo_args.x,v
retrieving revision 1.4
diff -u -w -b -r1.4 getdeviceinfo_args.x
--- dotx.d/getdeviceinfo_args.x	16 Nov 2007 15:18:17 -0000	1.4
+++ dotx.d/getdeviceinfo_args.x	25 Jan 2008 23:13:51 -0000
@@ -2,6 +2,6 @@
 	deviceid4		gdia_device_id;
 	layouttype4		gdia_layout_type;
 	count4			gdia_maxcount;
-	notification_mask4	gdia_notify_types;
+	bitmap4		gdia_notify_types;
 };
 
Index: dotx.d/getdeviceinfo_res.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/getdeviceinfo_res.x,v
retrieving revision 1.8
diff -u -w -b -r1.8 getdeviceinfo_res.x
--- dotx.d/getdeviceinfo_res.x	16 Dec 2007 18:32:42 -0000	1.8
+++ dotx.d/getdeviceinfo_res.x	25 Jan 2008 23:13:51 -0000
@@ -1,6 +1,6 @@
 struct GETDEVICEINFO4resok {
         device_addr4		gdir_device_addr;
-	notification_mask4	gdir_notification;
+	bitmap4		gdir_notification;
 };
 
 union GETDEVICEINFO4res switch (nfsstat4 gdir_status) {
Index: dotx.d/getdevicelist_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/getdevicelist_args.x,v
retrieving revision 1.4
diff -u -w -b -r1.4 getdevicelist_args.x
--- dotx.d/getdevicelist_args.x	16 Nov 2007 15:18:17 -0000	1.4
+++ dotx.d/getdevicelist_args.x	25 Jan 2008 23:13:51 -0000
@@ -1,8 +1,11 @@
 struct GETDEVICELIST4args {
+	/* CURRENT_FH: object belonging to the file system */
 	layouttype4		gdla_layout_type;
-	count4			gdla_maxcount;
+
+        /* number of deviceIDs to return */
+	count4		gdla_maxdevices;
+
 	nfs_cookie4		gdla_cookie;
 	verifier4		gdla_cookieverf;
-	notification_mask4	gdla_notify_types;
 };
 
Index: dotx.d/getdevicelist_res.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/getdevicelist_res.x,v
retrieving revision 1.8
diff -u -w -b -r1.8 getdevicelist_res.x
--- dotx.d/getdevicelist_res.x	16 Dec 2007 18:32:42 -0000	1.8
+++ dotx.d/getdevicelist_res.x	25 Jan 2008 23:13:51 -0000
@@ -1,16 +1,13 @@
 struct GETDEVICELIST4resok {
 	nfs_cookie4		gdlr_cookie;
 	verifier4		gdlr_cookieverf;
-	notification_mask4	gdlr_notification;
-	devlist_item4		gdlr_devinfo_list<>;
+	deviceid4		gdlr_deviceid_list<>;
 	bool			gdlr_eof;
 };
 
 union GETDEVICELIST4res switch (nfsstat4 gdlr_status) {
 case NFS4_OK:
 	GETDEVICELIST4resok	gdlr_resok4;
-case NFS4ERR_TOOSMALL:
-	count4 gdlr_mincount;
 default:
 	void;
 };
Index: dotx.d/head.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/head.x,v
retrieving revision 1.53
diff -u -w -b -r1.53 head.x
--- dotx.d/head.x	16 Dec 2007 21:07:55 -0000	1.53
+++ dotx.d/head.x	25 Jan 2008 23:13:51 -0000
@@ -355,8 +355,6 @@
 include(type_deviceid4.x)
 include(type_device_addr4.x)
 
-typedef bitmap4 device_notification_mask4;
-
 include(type_devlist_item4.x)
 include(type_layoutupdate4.x)
 include(type_layoutreturn4.x)
Index: dotx.d/spit_types.sh
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/spit_types.sh,v
retrieving revision 1.41
diff -u -w -b -r1.41 spit_types.sh
--- dotx.d/spit_types.sh	18 Dec 2007 16:56:06 -0000	1.41
+++ dotx.d/spit_types.sh	25 Jan 2008 23:13:51 -0000
@@ -457,16 +457,6 @@
 EOF
         ;;
 
-	type_devlist_item4.x )
-
-cat << EOF > $i
-struct devlist_item4 {
-        deviceid4                 dli_id;
-        device_addr4              dli_device_addr;
-};
-EOF
-	;;
-
 	type_client_owner4.x )
 
 cat << EOF > $i

---

From: Benny Halevy <[email-masked]>
Date: Sat, 26 Jan 2008 11:20:23 +0200

Mike, one nit:

...
               ^^^^^^^^^^^
"that of the" maybe?

Other than that the changes look good.  I like the fact that it 
simplifies the protocol overall!

What I found missing though is:

1. Description of how to cancel device notification. This is done by 
sending a GETDEVICEINFO
with null gdia_notify_types, right?  If the client has no reference to 
the device ID anymore and it just
wants to evict it out of its device cache, cancel the notification, and 
forget anything about it, can it
set gdia_maxcount to a small value allowing it to get a device_addr4 
with an empty da_addr_body
without getting a NFS4ERR_TOOSMALL error?

2. I'd consider the title for the GETDEVICELIST operation from
"Get All Device Mappings" to "Get Device Mappings for a File System"

"All" is misleading as the scope of the operation is the cfh's fsid.

3. I don't see NFS4ERR_NOT_SAME in GETDEVICELIST's valid error codes.
Should it be allowed to return when the list changes while getting it 
(using gdl[ar]_cookieverf
to detect that)?

Benny

Mike Eisler wrote:
> ------------------------------------------------------------------------
>

---

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 26 Jan 2008 12:10:29 -0500

Two nits.

The section title below needs to be fixed:

<section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14:
CB_NOTIFY_DEVICEID - Notify directory changes" >   

and "three" should be changed to "two" in

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]] 
Sent: Friday, January 25, 2008 6:37 PM
To: [email-masked]
Subject: Re: [nfsv4] Notes from Conference Call 

Attached.

---

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 26 Jan 2008 12:28:55 -0500

> can it set gdia_maxcount to a small value allowing 
> it to get a device_addr4 with an empty da_addr_body 

I think it would make sense to allow a gdia_maxcount of zero to indicate
that the caller doesn't want the information and so TOOSMALL, which is
based on the idea the client wants the information but has not allocated
enough space, would not have to be returned. 

-----Original Message-----
From: Benny Halevy [mailto:[email-masked]] 
Sent: Saturday, January 26, 2008 4:20 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [nfsv4] Notes from Conference Call

Mike, one nit:

+      belongs to the file system (i.e. all file objects sharing the
same

1. Description of how to cancel device notification. This is done by
sending a GETDEVICEINFO with null gdia_notify_types, right?  If the
client has no reference to the device ID anymore and it just wants to
evict it out of its device cache, cancel the notification, and forget
anything about it, can it set gdia_maxcount to a small value allowing it
to get a device_addr4 with an empty da_addr_body without getting a
NFS4ERR_TOOSMALL error?

2. I'd consider the title for the GETDEVICELIST operation from "Get All
Device Mappings" to "Get Device Mappings for a File System"

3. I don't see NFS4ERR_NOT_SAME in GETDEVICELIST's valid error codes.
Should it be allowed to return when the list changes while getting it
(using gdl[ar]_cookieverf to detect that)?

Benny

---

From: Mike Eisler <[email-masked]>
Date: Sat, 26 Jan 2008 10:52:11 -0800 (PST)

> without getting a NFS4ERR_TOOSMALL error?

Agreed.

> (using gdl[ar]_cookieverf

Agreed.

The error lists for both GETDEV* operations has issues, and so I made
changes.

New diffs attached (incorporating comments from Dave and Benny).
Index: Makefile
diff -u -b -w -r1.107 Makefile
+++ Makefile	26 Jan 2008 18:51:15 -0000
diff -u -b -w -r1.42 nfsv41_middle_datatypes.xml
+++ nfsv41_middle_datatypes.xml	26 Jan 2008 18:51:15 -0000
@@ -342,17 +342,6 @@
       </t>
     </section>
 
-    <section toc="exclude" anchor="devlist_item4"
-	     title="devlist_item4">
-	 <?rfc include='autogen/type_devlist_item4.xml'?>
-      <t>
-        An array of these values is returned by the GETDEVICELIST
-	operation.  They define the set of devices associated with a
-	file system for the layout type specified in the
-	GETDEVICELIST4args.
-      </t>
-    </section>
-
     <section toc="exclude" anchor="layout_content4" title="layout_content4">
 	 <?rfc include='autogen/type_layout_content4.xml'?>
       <t>
Index: nfsv41_middle_errors.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_errors.xml,v
retrieving revision 1.54
diff -u -b -w -r1.54 nfsv41_middle_errors.xml
--- nfsv41_middle_errors.xml	17 Dec 2007 22:17:39 -0000	1.54
+++ nfsv41_middle_errors.xml	26 Jan 2008 18:51:16 -0000
@@ -1917,15 +1917,13 @@
       NFS4ERR_NOFILEHANDLE,
       NFS4ERR_NOTSUPP,
       NFS4ERR_OP_NOT_IN_SESSION,
-      NFS4ERR_RECALLCONFLICT,
       NFS4ERR_REP_TOO_BIG,
       NFS4ERR_REP_TOO_BIG_TO_CACHE,
       NFS4ERR_REQ_TOO_BIG,
       NFS4ERR_SERVERFAULT,
       NFS4ERR_TOOSMALL,
       NFS4ERR_TOO_MANY_OPS,
-      NFS4ERR_UNKNOWN_LAYOUTTYPE,
-      NFS4ERR_UNSAFE_COMPOUND
+      NFS4ERR_UNKNOWN_LAYOUTTYPE
     </c>
 
     <c />
@@ -1937,19 +1935,19 @@
       NFS4ERR_BAD_COOKIE,
       NFS4ERR_DEADSESSION,
       NFS4ERR_DELAY,
+      NFS4ERR_FHEXPIRED,
       NFS4ERR_INVAL,
+      NFS4ERR_IO,
       NFS4ERR_NOFILEHANDLE,
       NFS4ERR_NOTSUPP,
+      NFS4ERR_NOT_SAME,
       NFS4ERR_OP_NOT_IN_SESSION,
-      NFS4ERR_RECALLCONFLICT,
       NFS4ERR_REP_TOO_BIG,
       NFS4ERR_REP_TOO_BIG_TO_CACHE,
       NFS4ERR_REQ_TOO_BIG,
       NFS4ERR_SERVERFAULT,
-      NFS4ERR_TOOSMALL,
       NFS4ERR_TOO_MANY_OPS,
-      NFS4ERR_UNKNOWN_LAYOUTTYPE,
-      NFS4ERR_UNSAFE_COMPOUND
+      NFS4ERR_UNKNOWN_LAYOUTTYPE
     </c>
 
     <c />
Index: nfsv41_middle_op_cb_notify_deviceid.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_cb_notify_deviceid.xml,v
retrieving revision 1.5
diff -u -b -w -r1.5 nfsv41_middle_op_cb_notify_deviceid.xml
--- nfsv41_middle_op_cb_notify_deviceid.xml	22 Dec 2007 18:56:39 -0000	1.5
+++ nfsv41_middle_op_cb_notify_deviceid.xml	26 Jan 2008 18:51:16 -0000
@@ -1,9 +1,9 @@
 <!-- 	$Id: nfsv41_middle_op_cb_notify_deviceid.xml,v 1.5 2007/12/22 18:56:39 mre Exp $	 -->
 <!-- Copyright (C) The IETF Trust (2007) -->
 <!-- Copyright (C) The Internet Society (2006) -->
-<section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14: CB_NOTIFY_DEVICEID - Notify directory changes" >
+<section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14: CB_NOTIFY_DEVICEID - Notify device ID changes" >
   <t>
-    Tell the client of directory changes.
+    Tell the client of deviceID changes.
   </t>

       <list style="hanging">
 
 	<t hangText="NOTIFY_DEVICEID4_CHANGE"><vspace />
 	  A previously provided device ID to device
Index: nfsv41_middle_op_getdeviceinfo.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdeviceinfo.xml,v
retrieving revision 1.16
diff -u -b -w -r1.16 nfsv41_middle_op_getdeviceinfo.xml
--- nfsv41_middle_op_getdeviceinfo.xml	17 Dec 2007 22:17:39 -0000	1.16
+++ nfsv41_middle_op_getdeviceinfo.xml	26 Jan 2008 18:51:16 -0000
@@ -20,7 +20,8 @@
       GETDEVICEINFO4resok structure and includes the XDR overhead.
       The server may return less data.  If the server is unable to
       return the information within the gdia_maxcount limit, the error
-      NFS4ERR_TOOSMALL will be returned.
+      NFS4ERR_TOOSMALL will be returned. However, if gdia_maxcount is
+      zero, NFS4ERR_TOOSMALL MUST NOT be returned.
     </t>
     <t>
 
@@ -34,19 +35,30 @@
       target="fattr4" />).  The numbers of bit positions
       are listed in the notify_device_type4 enumeration type
       (<xref target="OP_CB_NOTIFY_DEVICEID" />). Only
-      three enumerated values of notify_type4 currently
+      two enumerated values of notify_device_type4 currently
       apply to GETDEVICEINFO (and GETDEVICELIST,
       see <xref target="OP_GETDEVICELIST" />):
-      NOTIFY_DEVICEID4_ADD NOTIFY_DEVICEID4_CHANGE,
+      NOTIFY_DEVICEID4_CHANGE
       and NOTIFY_DEVICEID4_DELETE (see <xref
       target="OP_CB_NOTIFY_DEVICEID" />).
 
-
     </t>
     <t>
-      If an invalid device ID is given in gdia_device_id,
-      the server returns NFS4ERR_INVAL. <cref>mre: _INVAL
-      isn't the right error for an unknown device ID</cref>
+      The notification bitmap applies only to the specified device ID.
+      If a client issues GETDEVICEINFO on a deviceID multiple times,
+      the last notification bitmap is used by the server for
+      subsequent notifications. If the bitmap is zero or empty,
+      then notification are turned off on the device ID.
+    </t>
+    <t>
+     If the client wants to just update or turn off notifications,
+     it MAY issue GETDEVICEINFO with gdia_maxcount set to zero.
+     In that event, if the device ID is valid, the da_addr_body
+     field of the gdir_device_addr field will be of zero length.
+    </t>
+    <t>
+      If an unknown device ID is given in gdia_device_id,
+      the server returns NFS4ERR_NOENT.
 
       Otherwise, the device address
       information is returned in gdir_device_addr. 
@@ -64,6 +76,11 @@
   </section>
   <section toc="exclude" anchor="OP_GETDEVICEINFO_IMPLEMENTATION" title="IMPLEMENTATION">
   <t>
+   Aside from updating or turning off notifications, another
+   use case for gdia_maxcount being set to zero is to validate
+   a device ID.
+  </t>
+  <t>
     The client SHOULD request a notification for changes or
     deletion of a device ID to device address mapping so
     that the server can allow the client gracefully use a
@@ -108,35 +125,10 @@
     </t>
     
     <t>
      
Index: nfsv41_middle_op_getdevicelist.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdevicelist.xml,v
retrieving revision 1.15
diff -u -b -w -r1.15 nfsv41_middle_op_getdevicelist.xml
--- nfsv41_middle_op_getdevicelist.xml	14 Dec 2007 18:17:32 -0000	1.15
+++ nfsv41_middle_op_getdevicelist.xml	26 Jan 2008 18:51:16 -0000
@@ -1,7 +1,7 @@
 <!-- 	$Id: nfsv41_middle_op_getdevicelist.xml,v 1.15 2007/12/14 18:17:32 dnoveck Exp $	 -->
 <!-- Copyright (C) The IETF Trust (2007) -->
 <!-- Copyright (C) The Internet Society (2006) -->
-<section anchor="OP_GETDEVICELIST" title="Operation 48: GETDEVICELIST - Get All Device Mappings" >
+<section anchor="OP_GETDEVICELIST" title="Operation 48: GETDEVICELIST - Get All Device Mappings for a File System" >
 
   <section toc="exclude" anchor="OP_GETDEVICELIST_ARGUMENT" title="ARGUMENT">
    <?rfc include='autogen/getdevicelist_args.xml'?>
@@ -12,82 +12,35 @@
+      fsid as the that of current filehandle), and the layout type
     
     <t>
diff -u -b -w -r1.43 Makefile
+++ dotx.d/Makefile	26 Jan 2008 18:51:16 -0000
diff -u -b -w -r1.11 cb_notify_args.x
+++ dotx.d/cb_notify_args.x	26 Jan 2008 18:51:17 -0000
 
 struct notify4 {
-	/* composed from notify_type4 */
+	/* composed from notify_type4 or notify_deviceid_type4 */
         bitmap4         notify_mask;
         notifylist4     notify_vals;
 };
Index: dotx.d/cb_notify_deviceid_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/cb_notify_deviceid_args.x,v
retrieving revision 1.2
diff -u -b -w -r1.2 cb_notify_deviceid_args.x
--- dotx.d/cb_notify_deviceid_args.x	10 Dec 2007 19:59:04 -0000	1.2
+++ dotx.d/cb_notify_deviceid_args.x	26 Jan 2008 18:51:17 -0000
@@ -2,15 +2,14 @@
  * Device notification types.

  */

 };

 };

Index: dotx.d/get_dir_delegation_args.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/get_dir_delegation_args.x,v
retrieving revision 1.11
diff -u -b -w -r1.11 get_dir_delegation_args.x
--- dotx.d/get_dir_delegation_args.x	10 Dec 2007 09:27:14 -0000	1.11
+++ dotx.d/get_dir_delegation_args.x	26 Jan 2008 18:51:17 -0000
@@ -1,14 +1,10 @@
-/*
- * Mask of notification types.
- */
-typedef bitmap4 notification_mask4;

diff -u -b -w -r1.7 get_dir_delegation_res.x
+++ dotx.d/get_dir_delegation_res.x	26 Jan 2008 18:51:17 -0000
diff -u -b -w -r1.4 getdeviceinfo_args.x
+++ dotx.d/getdeviceinfo_args.x	26 Jan 2008 18:51:17 -0000
 
diff -u -b -w -r1.8 getdeviceinfo_res.x
+++ dotx.d/getdeviceinfo_res.x	26 Jan 2008 18:51:17 -0000
 
diff -u -b -w -r1.4 getdevicelist_args.x
+++ dotx.d/getdevicelist_args.x	26 Jan 2008 18:51:17 -0000
 
diff -u -b -w -r1.8 getdevicelist_res.x
+++ dotx.d/getdevicelist_res.x	26 Jan 2008 18:51:17 -0000
 
 union GETDEVICELIST4res switch (nfsstat4 gdlr_status) {
 case NFS4_OK:
 	GETDEVICELIST4resok	gdlr_resok4;
-case NFS4ERR_TOOSMALL:
-	count4 gdlr_mincount;
 default:
 	void;
 };
Index: dotx.d/head.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/head.x,v
retrieving revision 1.53
diff -u -b -w -r1.53 head.x
--- dotx.d/head.x	16 Dec 2007 21:07:55 -0000	1.53
+++ dotx.d/head.x	26 Jan 2008 18:51:17 -0000
@@ -355,8 +355,6 @@
 include(type_deviceid4.x)
 include(type_device_addr4.x)
 
diff -u -b -w -r1.41 spit_types.sh
+++ dotx.d/spit_types.sh	26 Jan 2008 18:51:17 -0000
@@ -457,16 +457,6 @@
 EOF
         ;;
 
-	type_devlist_item4.x )
-
-cat << EOF > $i
-struct devlist_item4 {
-        deviceid4                 dli_id;
 
 cat << EOF > $i

---

From: "J. Bruce Fields" <[email-masked]>
Date: Sat, 26 Jan 2008 14:41:56 -0500

> +      then notification are turned off on the device ID.

s/notification/notifications/

...

How about making that last line:

	use SAN devices; in such environments it may be helpful for

> +      in gdia_layout_type.  Since

Will it really help anyone to define "file system" at this point in the
spec?  I'd be tempted just to cut the parenthetical remark.

> +      The client provides gdla_maxdevices to limit the number of device IDs

s/for the result/in the result/.  Or just "limit the number of device
IDs returned."

--b.

---

From: [email-masked]
Date: Sat, 26 Jan 2008 18:40:36 -0500

device address mappings are no longer relevant for GETDEVICELIST

s/device ID to device address mappings/devices IDs/
s/in the mappings/in the list/

-----Original Message-----
From: Mike Eisler [mailto:[email-masked]]
Sent: Sat 1/26/2008 1:52 PM
To: [email-masked]
Subject: Re: [nfsv4] Notes from Conference Call

Agreed.

Agreed.

---

From: [email-masked]
Date: Sat, 26 Jan 2008 19:02:37 -0500

Unless I misread the patches, I think some additional changes are required in sections 12.3 and 12.6.

12.3
 
      GETDEVICELIST  (Section 18.41), allows clients to fetch all of the
      mappings of device IDs to storage device addresses for a specific
      file system.

should be

      GETDEVICELIST  (Section 18.41), allows clients to fetch all of the
      device IDs for a specific
      file system.

(or similar).

And in section 12.6

   When the client wants to send an I/O, it determines which device ID
   it needs to send the I/O command to by examining the data description
   in the layout.  It then sends a GETDEVICELIST to return a list of all
   device ID to device address mappings, or a GETDEVICEINFO to find the
   device address(es) of the device ID.  The client then sends the I/O
   request one of device ID's device addresses, using the storage
   protocol defined for the layout type.  Note that if a client has
   multiple I/Os to send, these I/O requests may be done in parallel.

s/GETDEVICELIST .* mappings, or a//

-Jason

-----Original Message-----
From: J. Bruce Fields [mailto:[email-masked]]
Sent: Sat 1/26/2008 2:41 PM
To: Mike Eisler
Cc: [email-masked]
Subject: Re: [nfsv4] Notes from Conference Call

...

--b.

---

From: Mike Eisler <[email-masked]>
Date: Sat, 26 Jan 2008 16:56:24 -0800 (PST)

> >        a client to determine device accessibility upon first file

Instead, I've moved the text to the IMPLEMENTATION section and wrote:

  An example of the use of this operation is for pNFS
 clients and servers that use LAYOUT4_BLOCK_VOLUME
 layouts.  In these environments it may be helpful
 for a client to determine device accessibility upon
 first file system access.

> spec?  I'd be tempted just to cut the parenthetical remark.

This turns out to be an excellent question, because there is no
formal definition of file system. So I added one. Then I noticed that
the definitions were in a odd place in the intro chapter, so I moved
them to be earlier in the intro chapter. Then, while looking at the
fsid description in the attribute chapter, I noticed that some of
the attributes were out of place. Then I noticed that we are using
"mandatory" as if it is an IETF keyword, and fixed all that in the
attributes chapter and other chapters. Goodbye MANDATORY (you never
existed) hello REQUIRED.

Regardless the parenthetical remark makes it clear that the server is
to
return all device IDs associated with the fsid of the file object
of the current filehandle, as opposed to some custom definition of
file system by some server implementor. 

I've made all your other changes as well as the ones Jason sent today.

New diffs attached.

Index: Makefile
===================================================================
RCS file: /cvs/stdk/NFSv41/Makefile,v
retrieving revision 1.107
diff -u -b -w -r1.107 Makefile
--- Makefile	31 Dec 2007 22:51:34 -0000	1.107
+++ Makefile	27 Jan 2008 00:37:31 -0000
@@ -1,3 +1,4 @@
+# Copyright (C) The IETF Trust (2007-2008)
 #
 # Manage the .xml for the NFSv4 minor version document.
 #
@@ -121,7 +122,6 @@
 		dotx.d/type_ssv_seal_plain_tkn4.x \
 		dotx.d/type_ssv_seal_cipher_tkn4.x \
 		dotx.d/type_layoutreturn4.x \
-		dotx.d/type_devlist_item4.x \
 		dotx.d/type_client_owner4.x \
 		dotx.d/type_server_owner4.x \
 		dotx.d/type_device_addr4.x \
@@ -186,7 +186,6 @@
 		autogen/type_ssv_seal_plain_tkn4.xml \
 		autogen/type_ssv_seal_cipher_tkn4.xml \
 		autogen/type_layoutreturn4.xml \
-		autogen/type_devlist_item4.xml \
 		autogen/type_client_owner4.xml \
 		autogen/type_server_owner4.xml \
 		autogen/type_device_addr4.xml \
Index: nfsv41_middle_caching.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_caching.xml,v
retrieving revision 1.28
diff -u -b -w -r1.28 nfsv41_middle_caching.xml
--- nfsv41_middle_caching.xml	31 Dec 2007 16:13:16 -0000	1.28
+++ nfsv41_middle_caching.xml	27 Jan 2008 00:37:32 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_caching.xml,v 1.28 2007/12/31 16:13:16 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="Client-Side Caching" >
   <t>
@@ -1165,8 +1165,8 @@
              anchor="via_want_delegation" >
       <t>
         In addition to providing delegations as part of the response 
-        to OPEN operations, servers may optionally provide delegations 
-        separate from open, via the WANT_DELEGATION operation.  This 
+        to OPEN operations, servers MAY provide delegations 
+        separate from open, via the OPTIONAL WANT_DELEGATION operation.  This 
         allows delegations to be obtained in advance of an OPEN that 
         might benefit from them, for objects which are not a valid target
         of OPEN, or to deal with cases in which a 
Index: nfsv41_middle_core_infrastructure.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_core_infrastructure.xml,v
retrieving revision 1.78
diff -u -b -w -r1.78 nfsv41_middle_core_infrastructure.xml
--- nfsv41_middle_core_infrastructure.xml	16 Jan 2008 04:57:37 -0000	1.78
+++ nfsv41_middle_core_infrastructure.xml	27 Jan 2008 00:37:33 -0000
@@ -1,5 +1,5 @@
 <!--    $Id: nfsv41_middle_core_infrastructure.xml,v 1.78 2008/01/16 04:57:37 mre Exp $  -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (1996-2006)  -->
 
 <section anchor="Core Infrastructure" title="Core Infrastructure">
@@ -114,8 +114,8 @@
       Each mandated mechanism specifies minimum set of cryptographic
       algorithms for implementing integrity and privacy. NFSv4.1
       clients and servers MUST be implemented on operating environments
-      that comply with the mandatory cryptographic algorithms
-      of each mandated mechanism.
+      that comply with the REQUIRED cryptographic algorithms
+      of each REQUIRED mechanism.
      </t>
 
       <section anchor="kerberosv5" title="Kerberos V5">
@@ -908,7 +908,7 @@
     supported by the security policy.  But in practice, the
     client may start looking for strong flavors from those
     supported by the security policy, followed by those in
-    the mandatory set.
+    the REQUIRED set.
    </t>
    <t>
     The NFSv4.1 server MUST NOT return NFS4ERR_WRONGSEC to a
@@ -1084,9 +1084,9 @@
  </t>
 
  <t>
- Minor versions may declare an operation as mandatory to NOT implement.
+ Minor versions may declare an operation as REQUIRED to NOT implement.
  <vspace blankLines='1' />
- Specifying an operation as "mandatory to not implement" is equivalent
+ Specifying an operation as "REQUIRED to NOT implement" is equivalent
  to obsoleting an operation.  For the client, it means that the
  operation should not be sent to the server.  For the server, an NFS
  error can be returned as opposed to "dropping" the request as an XDR
@@ -1094,23 +1094,23 @@
  operation while maintaining its structure so that a future minor version can reintroduce the operation.
  <list style='numbers'>
  <t>
- Minor versions may declare attributes mandatory to NOT implement.
+ Minor versions may declare attributes REQUIRED to NOT implement.
  </t>
  <t>
  Minor versions may declare flag bits or enumeration values as
- mandatory to NOT implement.
+ REQUIRED to NOT implement.
  </t>
  </list>
  </t>
 
  <t>
- Minor versions may downgrade features from mandatory to recommended,
- or recommended to optional.
+ Minor versions may downgrade features from REQUIRED to RECOMMENDED,
+ or RECOMMENDED to OPTIONAL.
  </t>
 
  <t>
- Minor versions may upgrade features from optional to recommended or
- recommended to mandatory.
+ Minor versions may upgrade features from OPTIONAL to RECOMMENDED or
+ RECOMMENDED to REQUIRED.
  </t>
 
  <t>
@@ -1119,13 +1119,13 @@
 
  <t>
  Except for infrastructural changes, no new features may be
- introduced as mandatory in a minor version.
+ introduced as REQUIRED in a minor version.
  <vspace blankLines='1' />
  This rule allows for the introduction of new functionality and forces
  the use of implementation experience before designating a feature as
- mandatory. On the other hand, some classes of features are
+ REQUIRED. On the other hand, some classes of features are
  infrastructural and have broad effects. Allowing such features
- to not be mandatory complicates implementation of the minor version.
+ to not be REQUIRED complicates implementation of the minor version.
  </t>
 
  <t>
@@ -1188,7 +1188,7 @@
  <section anchor="Transport Layers" title="Transport Layers">
 
   <section anchor="Required and Recommended Transport Attributes"
-   title="Required and Recommended Properties of Transports">
+   title="Required and RECOMMENDED Properties of Transports">
   <t>
    NFSv4.1 works over RDMA and non-RDMA_based transports with
    the following attributes: 
@@ -3156,17 +3156,23 @@
      of a transport connection that has previously been
      associated with a session. If the goal of a counter
      threat strategy is to prevent connection hijacking,
-     the use of IPsec is RECOMMENDED.  If the goal of
+     the use of IPsec is RECOMMENDED.
+   </t>
+   <t>
+     If the goal of
      a counter threat strategy is to prevent a connection
      hijacker from making unauthorized state changes,
      then the SP4_MACH_CRED protection approach
      can be used with a client ID per user (i.e. the
      aforementioned third scenario for machine credential
-     state protection). Each EXCHANGE_ID can specify the
+     state protection). For each unique user, the client
+     invokes EXCHANGE_ID with the user's credential, specifying
+     SP4_MACH_CRED, and that
      all operations MUST be protected with the machine
      credential.
      The server will then reject any
-     subsequent operations on the client ID that do not
+     subsequent operations on the client ID or its sessions
+     that do not
      use RPCSEC_GSS with privacy or integrity and do not
      use the same credential that created the client ID.
    </t>
Index: nfsv41_middle_datatypes.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_datatypes.xml,v
retrieving revision 1.42
diff -u -b -w -r1.42 nfsv41_middle_datatypes.xml
--- nfsv41_middle_datatypes.xml	22 Dec 2007 18:56:39 -0000	1.42
+++ nfsv41_middle_datatypes.xml	27 Jan 2008 00:37:33 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_datatypes.xml,v 1.42 2007/12/22 18:56:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="Protocol Constants and Data Types">
   <t>
@@ -89,7 +89,7 @@
 	when processing time values, preserving as much accuracy as
 	possible. If the precision of timestamps stored for a
 	file system object is less than defined, loss of precision can
-	occur.  An adjunct time maintenance protocol is recommended to
+	occur.  An adjunct time maintenance protocol is RECOMMENDED to
 	reduce client and server time skew.
       </t>
     </section>
@@ -123,7 +123,7 @@
 	 <?rfc include='autogen/type_chg_policy4.xml'?>
       <t>
          The chg_policy4 data type is used for the change_policy
-         recommended attribute.  It provides change sequencing indication
+         RECOMMENDED attribute.  It provides change sequencing indication
          analogous to the change attribute.  To enable the server to 
          present a value valid across server re-initialization without
          requiring persistent storage, two 64-bit quantities are used,
@@ -342,17 +342,6 @@
       </t>
     </section>
 
-    <section toc="exclude" anchor="devlist_item4"
-	     title="devlist_item4">
-	 <?rfc include='autogen/type_devlist_item4.xml'?>
-      <t>
-        An array of these values is returned by the GETDEVICELIST
-	operation.  They define the set of devices associated with a
-	file system for the layout type specified in the
-	GETDEVICELIST4args.
-      </t>
-    </section>
-
     <section toc="exclude" anchor="layout_content4" title="layout_content4">
 	 <?rfc include='autogen/type_layout_content4.xml'?>
       <t>
@@ -432,7 +421,7 @@
 	implementation detail.  The nii_domain field is the DNS domain
 	name that the implementer is associated with.  The nii_name
 	field is the product name of the implementation and is
-	completely free form.  It is recommended that the nii_name be
+	completely free form.  It is RECOMMENDED that the nii_name be
 	used to distinguish machine architecture, machine platforms,
 	revisions, versions, and patch levels.  The nii_date field is
 	the timestamp of when the software instance was published or
@@ -467,22 +456,22 @@
   
         <c>threshold4_read_size</c><c>0</c><c>length4</c>
         <c>
-           The file size below which it is recommended to read data
+           The file size below which it is RECOMMENDED to read data
   	 through the MDS.
         </c>
         <c>threshold4_write_size</c><c>1</c><c>length4</c>
         <c>
-           The file size below which it is recommended to write data
+           The file size below which it is RECOMMENDED to write data
   	 through the MDS.
         </c>
         <c>threshold4_read_iosize</c><c>2</c><c>length4</c>
         <c>
-          For read I/O sizes below this threshold it is recommended to
+          For read I/O sizes below this threshold it is RECOMMENDED to
   	read data through the MDS
         </c>
         <c>threshold4_write_iosize</c><c>3</c><c>length4</c>
         <c>
-          For write I/O sizes below this threshold it is recommended to
+          For write I/O sizes below this threshold it is RECOMMENDED to
   	write data through the MDS
         </c>
       </texttable>      
Index: nfsv41_middle_errors.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_errors.xml,v
retrieving revision 1.54
diff -u -b -w -r1.54 nfsv41_middle_errors.xml
--- nfsv41_middle_errors.xml	17 Dec 2007 22:17:39 -0000	1.54
+++ nfsv41_middle_errors.xml	27 Jan 2008 00:37:33 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_errors.xml,v 1.54 2007/12/17 22:17:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 
 <section title="Error Values">
@@ -512,8 +512,8 @@
              anchor="err_NOTSUPP">
       <t>
         Operation not supported, either because the operation is
-        an optional one and is not supported by this server or
-        because the operation is mandatory to not implement in 
+        an OPTIONAL one and is not supported by this server or
+        because the operation is REQUIRED to NOT implement in 
         the current minor version.
       </t>
     </section>
@@ -1538,7 +1538,7 @@
     by all operations with two important exceptions:
     <list style="symbols">
       <t>
-        The operations which are mandatory to not implement: 
+        The operations which MUST NOT be implemented: 
         OPEN_CONFIRM, RELEASE_LOCKOWNER, RENEW, SETCLIENTID, and
         SETCLIENTID_CONFIRM.
       </t>
@@ -1917,15 +1917,13 @@
       NFS4ERR_NOFILEHANDLE,
       NFS4ERR_NOTSUPP,
       NFS4ERR_OP_NOT_IN_SESSION,
-      NFS4ERR_RECALLCONFLICT,
       NFS4ERR_REP_TOO_BIG,
       NFS4ERR_REP_TOO_BIG_TO_CACHE,
       NFS4ERR_REQ_TOO_BIG,
       NFS4ERR_SERVERFAULT,
       NFS4ERR_TOOSMALL,
       NFS4ERR_TOO_MANY_OPS,
-      NFS4ERR_UNKNOWN_LAYOUTTYPE,
-      NFS4ERR_UNSAFE_COMPOUND
+      NFS4ERR_UNKNOWN_LAYOUTTYPE
     </c>
 
     <c />
@@ -1937,19 +1935,19 @@
       NFS4ERR_BAD_COOKIE,
       NFS4ERR_DEADSESSION,
       NFS4ERR_DELAY,
+      NFS4ERR_FHEXPIRED,
       NFS4ERR_INVAL,
+      NFS4ERR_IO,
       NFS4ERR_NOFILEHANDLE,
       NFS4ERR_NOTSUPP,
+      NFS4ERR_NOT_SAME,
 
     <c />
Index: nfsv41_middle_file_layout.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_file_layout.xml,v
retrieving revision 1.40
diff -u -b -w -r1.40 nfsv41_middle_file_layout.xml
--- nfsv41_middle_file_layout.xml	20 Dec 2007 20:46:10 -0000	1.40
+++ nfsv41_middle_file_layout.xml	27 Jan 2008 00:37:34 -0000
@@ -1,5 +1,5 @@
 <!-- $Id: nfsv41_middle_file_layout.xml,v 1.40 2007/12/20 20:46:10 mre Exp $ -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="PNFS: NFSv4.1 File Layout Type" anchor="file_layout_type">
 <t>
@@ -11,7 +11,7 @@
 </t>
  <section title="Client ID and Session Considerations" anchor="pnfs_session_stuff">
  <t>
-  Sessions are a mandatory feature of NFSv4.1, and this
+  Sessions are a REQUIRED feature of NFSv4.1, and this
   extends to both the metadata server and file-based (NFSv4.1-based)
   data servers.
  </t>
@@ -113,7 +113,7 @@
    client ID of the data server, the data server has no
    obvious way to determine the metadata server from the
    COMPOUND procedure, and thus has no way to validate the
-   stateid. One recommended approach is for pNFS servers to
+   stateid. One RECOMMENDED approach is for pNFS servers to
    encode metadata server routing and/or identity
    information in the data server filehandles as returned
    in the layout.
@@ -1077,7 +1077,7 @@
  <t>
   Because of the complexity induced by assigning filehandles so
   they can be used on both a data server and a metadata server, it
-  is recommended that where the same server can have both
+  is RECOMMENDED that where the same server can have both
   personalities, the server assign separate unique filehandles
   to both personalities. This makes it unambiguous for which server
   a given request is intended.
Index: nfsv41_middle_fileattributes.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_fileattributes.xml,v
retrieving revision 1.39
diff -u -b -w -r1.39 nfsv41_middle_fileattributes.xml
--- nfsv41_middle_fileattributes.xml	19 Dec 2007 15:27:13 -0000	1.39
+++ nfsv41_middle_fileattributes.xml	27 Jan 2008 00:37:34 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_fileattributes.xml,v 1.39 2007/12/19 15:27:13 dnoveck Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="file_attributes" title="File Attributes">
   <t>
@@ -14,13 +14,13 @@
     attributes (or a subset thereof).
   </t>
   <t>
-    To this end, attributes are divided into three groups: mandatory,
-    recommended, and named.  Both mandatory and recommended attributes are
+    To this end, attributes are divided into three groups: REQUIRED,
+    RECOMMENDED, and named.  Both REQUIRED and RECOMMENDED attributes are
     supported in the NFSv4.1 protocol by a specific and well-defined
     encoding and are identified by number.  They are requested by setting
     a bit in the bit vector sent in the GETATTR request; the server
     response includes a bit vector to list what attributes were returned
-    in the response.  New mandatory or recommended attributes may be added
+    in the response.  New REQUIRED or RECOMMENDED attributes may be added
     to the NFS protocol between major revisions by publishing a
     standards-track RFC which allocates a new attribute number value and
     defines the encoding for the attribute.  See
@@ -49,15 +49,15 @@
   <t>
     Named attributes are intended for data needed by applications rather
     than by an NFS client implementation.  NFS implementors are strongly
-    encouraged to define their new attributes as recommended attributes by
+    encouraged to define their new attributes as RECOMMENDED attributes by
     bringing them to the IETF standards-track process.
   </t>
   <t>
-    The set of attributes which are classified as mandatory is
+    The set of attributes which are classified as REQUIRED is
     deliberately small since servers must do whatever it takes to support
-    them.  A server should support as many of the recommended attributes
+    them.  A server should support as many of the RECOMMENDED attributes
     as possible but by their definition, the server is not required to
-    support all of them.  Attributes are deemed mandatory if the data is
+    support all of them.  Attributes are deemed REQUIRED if the data is
     both needed by a large number of clients and is not otherwise
     reasonably computable by the client when support is not provided on
     the server.
@@ -70,19 +70,19 @@
     not.  Therefore, operations such as SETATTR and GETATTR on the named
     attribute directory are undefined.
   </t>
-  <section anchor="mandatory_attributes_intro" title="Mandatory Attributes">
+  <section anchor="mandatory_attributes_intro" title="REQUIRED Attributes">
     <t>
       These MUST be supported by every NFSv4.1 client and server in
       order to ensure a minimum level of interoperability.  The server must
       store and return these attributes and the client must be able to
       function with an attribute set limited to these attributes.  With just
-      the mandatory attributes some client functionality may be impaired or
+      the REQUIRED attributes some client functionality may be impaired or
       limited in some ways.  A client may ask for any of these attributes to
       be returned by setting a bit in the GETATTR request and the server
       must return their value.
     </t>
   </section>
-  <section anchor="recommended_attributes_intro" title="Recommended Attributes">
+  <section anchor="recommended_attributes_intro" title="RECOMMENDED Attributes">
     <t>
       These attributes are understood well enough to warrant support in the
       NFSv4.1 protocol.  However, they may not be supported on all
@@ -127,7 +127,7 @@
     <t>
       Named attributes and the named attribute directory may have have 
       their own (non-named) attributes.  Each of objects must have all 
-      of the mandatory attributes and may have additional recommended 
+      of the REQUIRED attributes and may have additional RECOMMENDED 
       attributes.  However, the set of attributes for named attributes 
       and the named attribute directory need not be as large as, and
       typically will not be as large as that for other objects in that 
@@ -142,7 +142,7 @@
       attribute directory.
     </t>
     <t>
-      It is recommended that servers support arbitrary named attributes.  A
+      It is RECOMMENDED that servers support arbitrary named attributes.  A
       client should not depend on the ability to store any named attributes
       in the server's file system.  If a server does support named
       attributes, a client which is also able to handle them should be able
@@ -192,7 +192,7 @@
   </section>
   <section title="Classification of Attributes">
     <t>
-      Each of the Mandatory and Recommended attributes can be classified in
+      Each of the REQUIRED and RECOMMENDED attributes can be classified in
       one of three categories: per server, per file system, or per file system
       object.  Note that it is possible that some per file system attributes
       may vary within the file system.  See the "homogeneous" attribute for
@@ -251,12 +251,12 @@
   </section>
 
   <section anchor="mandatory_attributes" 
-	   title="Mandatory Attributes - List and Definition References">
+	   title="REQUIRED Attributes - List and Definition References">
     <texttable>
       <ttcol align='left' >name</ttcol>
       <ttcol align='left' >Id</ttcol>
       <ttcol align='left' >Data Type</ttcol>
-      <ttcol align='left' >Acc.</ttcol>
+      <ttcol align='left' >Acc</ttcol>
       <ttcol align='left' >Defined in:</ttcol>
 
       <c>supported_attrs</c><c>0</c><c>bitmap4</c><c>RD</c>
@@ -332,12 +332,12 @@
     </texttable>
   </section>
   <section anchor="recommended_attributes" 
-	   title="Recommended Attributes - List and Definition References">
+	   title="RECOMMENDED Attributes - List and Definition References">
     <texttable>
       <ttcol align='left' >name</ttcol>
       <ttcol align='left' >Id</ttcol>
       <ttcol align='left' >Data Type</ttcol>
-      <ttcol align='left' >Acc.</ttcol>
+      <ttcol align='left' >Acc</ttcol>
       <ttcol align='left' >Defined in:</ttcol>
 
       <c>acl</c><c>12</c><c>nfsace4&lt;></c><c>R/W</c>
@@ -430,7 +430,7 @@
 	<xref target="attrdef_fs_locations" />
       </c>
 
-      <c>fs_locations_info</c><c>67</c><c></c><c>RD</c>
+      <c>fs_locations_info</c><c>67</c><c>*</c><c>RD</c>
       <c>
 	<xref target="attrdef_fs_locations_info" />
       </c>
@@ -662,27 +662,19 @@
       </c>
 
     </texttable>
+    <t>* fs_locations_info4</t>
   </section>
 
   <section anchor="attribute_definitions" title="Attribute
 						 Definitions">
 
+   <section anchor="required_attr" title="Definitions of REQUIRED Attributes">
+
     <section toc="exclude" anchor="attrdef_supp_attr" 
 	     title="Attribute 0: supported_attrs">
 	<t>
-	The bit vector which would retrieve all mandatory and
-	recommended attributes that are supported for this object.
-	The scope of this attribute applies to all objects with a
-	matching fsid.
-	</t>
-    </section>
-
-    <section toc="exclude" anchor="attrdef_suppattr_exclcreat" 
-	     title="Attribute 75: suppattr_exclcreat">
-	<t>
-	The bit vector which would set all mandatory and
-	recommended attributes that are supported by the EXCLUSIVE4_1
-        method of file creation via the OPEN operation.
+	The bit vector which would retrieve all REQUIRED and
+	RECOMMENDED attributes that are supported for this object.
 	The scope of this attribute applies to all objects with a
 	matching fsid.
 	</t>
@@ -766,7 +758,7 @@
     </section>
 
     <section toc="exclude" anchor="attrdef_size" 
-	     title="Attribute 3: size">
+	     title="Attribute 4: size">
 	<t>
 	  The size of the object in bytes.
 	</t>
@@ -799,7 +791,7 @@
 	<t>
 	  Unique file system identifier for the file system holding this
 	  object.  fsid contains major and minor components each of
-	  which are uint64_t.
+	  which are of data type uint64_t.
 	</t>
     </section>
 
@@ -832,6 +824,25 @@
 	</t>
     </section>
 
+    <section toc="exclude" anchor="attrdef_suppattr_exclcreat" 
+	     title="Attribute 75: suppattr_exclcreat">
+	<t>
+	The bit vector which would set all REQUIRED and
+	RECOMMENDED attributes that are supported by the EXCLUSIVE4_1
+        method of file creation via the OPEN operation.
+	The scope of this attribute applies to all objects with a
+	matching fsid.
+	</t>
+    </section>
+
+   </section>
+
+   <section anchor="recommended_attr" title="Definitions of Uncategorized RECOMMENDED Attributes">
+    <t>
+     The definitions of most of the RECOMMENDED attributes follow. Collections
+     that share a common category category are defined in other sections.
+    </t>
+
     <section toc="exclude" anchor="attrdef_archive"
 	     title="Attribute 14: archive">
       <t>
@@ -856,6 +867,13 @@
       </t>
     </section>
 
+    <section toc="exclude" anchor="attrdef_case_preserving"
+	     title="Attribute 17: case_preserving">
+      <t>
+	True, if filename case on this file system are preserved.
+      </t>
+    </section>
+
     <section toc="exclude" anchor="attrdef_change_policy"
 	     title="Attribute 60: change_policy">
       <t>
@@ -880,13 +898,6 @@
       </t>
     </section>
 
-    <section toc="exclude" anchor="attrdef_case_preserving"
-	     title="Attribute 17: case_preserving">
-      <t>
-	True, if filename case on this file system are preserved.
-      </t>
-    </section>
-
     <section toc="exclude" anchor="attrdef_chown_restricted"
 	     title="Attribute 18: chown_restricted">
       <t>
@@ -1294,10 +1305,12 @@
 
   </section>
 
+  </section>
+
   <section anchor="owner_owner_group" 
 	   title="Interpreting owner and owner_group">
     <t>
-      The recommended attributes "owner" and "owner_group" (and also
+      The RECOMMENDED attributes "owner" and "owner_group" (and also
       users and groups within the "acl" attribute) are represented in
       terms of a UTF-8 string.  To avoid a representation that is tied
       to a particular underlying implementation at the client or
@@ -1574,7 +1587,7 @@
       When retention is enabled, retention MUST extend to the data of
       the file, and the name of file. The server MAY extend retention
       any other property of the file, including any subset of
-      mandatory, recommended, and named attributes, with the
+      REQUIRED, RECOMMENDED, and named attributes, with the
       exceptions noted in this section.
     </t>
     <t>
@@ -1609,7 +1622,7 @@
     <section toc="exclude" anchor="attrdef_retention_set"
 	     title="Attribute 70: retention_set">
       <t>
-	This attributes is used to set the retention duration and
+	This attribute is used to set the retention duration and
 	optionally enable retention for the associated file object.
 	This attribute is only modifiable via SETATTR operation and
 	may not be read with the GETATTR operation.  This attribute
Index: nfsv41_middle_fileattributes_acls.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_fileattributes_acls.xml,v
retrieving revision 1.25
diff -u -b -w -r1.25 nfsv41_middle_fileattributes_acls.xml
--- nfsv41_middle_fileattributes_acls.xml	17 Dec 2007 22:17:39 -0000	1.25
+++ nfsv41_middle_fileattributes_acls.xml	27 Jan 2008 00:37:34 -0000
@@ -1,5 +1,5 @@
 <!--    $Id: nfsv41_middle_fileattributes_acls.xml,v 1.25 2007/12/17 22:17:39 mre Exp $  -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="Security Related Attributes" anchor="acl">
     <t>
@@ -244,13 +244,13 @@
             NFS4ERR_ATTRNOTSUPP.
           </t>
           <t>
-            Support for any of the ACL attributes is optional.
+            Support for any of the ACL attributes is RECOMMENDED.
             However, a server that supports either of the new ACL
-            attributes (dacl or sacl) must allow use of the new ACL
+            attributes (dacl or sacl) MUST allow use of the new ACL
             attributes to access all of the ACE types which it
-            supports.  In more detail: if such a server supports ALLOW
-            or DENY ACEs, then it must support the dacl attribute, and
-            if it supports AUDIT or ALARM ACEs, then it must support
+            supports.  In other words, if such a server supports ALLOW
+            or DENY ACEs, then it MUST support the dacl attribute, and
+            if it supports AUDIT or ALARM ACEs, then it MUST support
             the sacl attribute.
           </t>
         </section>
@@ -1511,7 +1511,7 @@
             modifications to inherited ACEs on descendents.
           </t>
           <t>
-            However, the dacl and sacl provide an optional mechanism
+            However, the dacl and sacl provide an OPTIONAL mechanism
             which allows a client application to propagate changes to
             inheritable ACEs to an entire directory hierarchy.
           </t>
Index: nfsv41_middle_filehandles.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_filehandles.xml,v
retrieving revision 1.10
diff -u -b -w -r1.10 nfsv41_middle_filehandles.xml
--- nfsv41_middle_filehandles.xml	20 Nov 2007 23:48:48 -0000	1.10
+++ nfsv41_middle_filehandles.xml	27 Jan 2008 00:37:35 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_filehandles.xml,v 1.10 2007/11/20 23:48:48 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="Filehandles" title="Filehandles">
   <t>
@@ -163,7 +163,7 @@
         return an error of NFS4ERR_FHEXPIRED.
       </t>
       <t>
-        The mandatory attribute "fh_expire_type" is used by the client to
+        The REQUIRED attribute "fh_expire_type" is used by the client to
         determine what type of filehandle the server is providing for a
         particular file system.  This attribute is a bitmask with the
         following values:
Index: nfsv41_middle_filelocking.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_filelocking.xml,v
retrieving revision 1.42
diff -u -b -w -r1.42 nfsv41_middle_filelocking.xml
--- nfsv41_middle_filelocking.xml	17 Dec 2007 22:17:39 -0000	1.42
+++ nfsv41_middle_filelocking.xml	27 Jan 2008 00:37:35 -0000
@@ -1,5 +1,5 @@
 <!--  $Id: nfsv41_middle_filelocking.xml,v 1.42 2007/12/17 22:17:39 mre Exp $       -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 
 <section title="File Locking and Share Reservations" anchor="file_locking">
@@ -277,7 +277,7 @@
     <t>
       Some clients require the support of blocking locks.  While NFSv4.1 
       provides a callback when a previously unavailable lock becomes 
-      available, this is an optional feature and clients cannot 
+      available, this is an OPTIONAL feature and clients cannot 
       depend on its presence.  Clients need to be prepared to continually 
       poll for the lock.  This
       presents a fairness problem.  Two new lock types are added, READW and
Index: nfsv41_middle_introduction.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_introduction.xml,v
retrieving revision 1.29
diff -u -b -w -r1.29 nfsv41_middle_introduction.xml
--- nfsv41_middle_introduction.xml	16 Jan 2008 04:57:37 -0000	1.29
+++ nfsv41_middle_introduction.xml	27 Jan 2008 00:37:35 -0000
@@ -1,5 +1,5 @@
 <!--  $Id: nfsv41_middle_introduction.xml,v 1.29 2008/01/16 04:57:37 mre Exp $       -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="intro" title="Introduction" >
   <section anchor="intro_the_protocol" title="The NFS Version 4 Minor Version 1 Protocol">
@@ -19,8 +19,8 @@
       operations and the RECLAIM_COMPLETE operation.
       These two new features are infrastructural in
       nature and simplify implementation of existing and
-      other new features.  Making them optional would
-      add undue complexity to protocol definition and
+      other new features.  Making them anything but REQUIRED
+      would add undue complexity to protocol definition and
       implementation.  NFSv4.1 accordingly updates the
       <xref target="minor_versioning">Minor Versioning
       guidelines</xref>.
@@ -115,8 +115,13 @@
           discovered in the base protocol.
         </t>
         <t>
-          To add clarity and specificity to areas left unaddressed or 
-          not addressed in sufficient detail in the base protocol. 
+          To add clarity and specificity to areas left
+          unaddressed or not addressed in sufficient
+          detail in the base protocol. However, as stated
+          in <xref target="scope_of_doc" />, it is not
+          a goal to clarify the NFSv4.0 protocol in the
+          NFSv4.1 specification.
+
         </t>
         <t>
           To add specific features based on experience with the existing
@@ -130,6 +135,134 @@
       </list>
     </t>
   </section>
+  <section anchor="intro_definitions" title="General Definitions">
+    <t>
+      The following definitions are provided for the purpose of providing
+      an appropriate context for the reader.
+      <list style='hanging'>
+        <t hangText="Byte" anchor="byte">
+          This document defines a byte as an octet, i.e. a datum
+          exactly 8 bits in length.
+        </t>
+        <t hangText="Client" anchor="client_def">
+          The "client" is the entity that accesses the NFS server's
+          resources.  The client may be an application which contains
+          the logic to access the NFS server directly.  The client
+          may also be the traditional operating system client that
+	  provides remote file system services for a set of applications.
+        <vspace blankLines='1' />
+          A client is uniquely identified by a Client Owner.
+        <vspace blankLines='1' />
+          With reference to file locking, the client is also the entity that
+          maintains a set of locks on behalf of one or more
+          applications.  This client is responsible for crash or
+          failure recovery for those locks it manages.
+        <vspace blankLines='1' />
+          Note that multiple clients may share the same transport and
+          connection and
+          multiple clients may exist on the same network node.
+        </t>
+        <t hangText="Client ID">
+          A 64-bit quantity used as a unique, short-hand reference to
+          a client supplied Verifier and client owner.  The server is
+          responsible for supplying the client ID.
+        </t>
+        <t hangText="Client Owner">
+          The client owner is a unique string, opaque to the server,
+          which identifies a client. Multiple network connections and source
+          network addresses originating from those connections may share
+          a client owner. The server is expected to treat requests
+          from connnections with the same client owner as coming from
+          the same client.
+        </t>
+
+        <t hangText="File System">
+          The collection of objects on a server (as
+          identified by the major identifier of a Server
+          Owner, which is defined later in this section),
+          that share the same fsid attribute (see <xref
+          target="attrdef_fsid"/>).
+
+        </t>
+        <t hangText="Lease">
+          An interval of time defined by the server for which the
+          client is irrevocably granted a lock.  At the end of a
+          lease period the lock may be revoked if the lease has not
+          been extended.  The lock must be revoked if a conflicting
+          lock has been granted after the lease interval.
+        <vspace blankLines='1' />
+          All leases granted by a server have the same fixed
+          interval.  Note that the fixed interval was chosen to
+          alleviate the expense a server would have in maintaining
+          state about variable length leases across server failures.
+        </t>
+        <t hangText="Lock">
+          The term "lock" is used to refer to record (byte-range)
+          locks, share reservations, delegations, or layouts unless
+          specifically stated otherwise.
+        </t>
+        <t hangText="Server">
+          The "Server" is the entity responsible for coordinating
+          client access to a set of file systems and is identified by a Server
+          owner. A server can span
+          multiple network addresses.
+        </t>
+        <t hangText="Server Owner">
+          The "Server Owner" identifies the server to the client.
+          The server owner consists of a major and minor identifier.
+          When the client has two connections each to a peer with the
+          same major identifier, the client assumes both peers are
+          the same server (the server namespace is the
+          same via each connection), and assumes and
+          lock state is sharable across both connections. When each peer
+          has both the same major and minor identifier, the client assumes
+          each connection might be associatable with the same session.
+        </t>
+        <t hangText="Stable Storage">
+          NFSv4.1 servers must be able to recover without data
+          loss from multiple power failures (including cascading
+          power failures, that is, several power failures in quick
+          succession), operating system failures, and hardware
+          failure of components other than the storage medium itself
+          (for example, disk, nonvolatile RAM).
+        <vspace blankLines='1' />
+          Some examples of stable storage that are allowable for an
+          NFS server include:
+          <list style='numbers'>
+             <t>
+               Media commit of data, that is, the modified data has
+               been successfully written to the disk media, for
+               example, the disk platter.
+             </t>
+             <t>
+               An immediate reply disk drive with battery-backed on-
+               drive intermediate storage or uninterruptible power
+               system (UPS).
+             </t>
+             <t>
+               Server commit of data with battery-backed intermediate
+               storage and recovery software.
+             </t>
+             <t>
+               Cache commit with uninterruptible power system (UPS) and
+               recovery software.
+             </t>
+           </list>
+        </t>
+        <t hangText="Stateid">
+          A 128-bit quantity returned by a server that uniquely
+          defines the open and locking state provided by the server
+          for a specific open or lock owner for a specific file and
+          type of lock.
+        </t>
+        <t hangText="Verifier">
+          A 64-bit quantity generated by the client that the server
+          can use to determine if the client has restarted and lost
+           all previous lock state.        
+        </t>
+      </list>
+    </t>
+  </section>
   <section anchor="feature overview" 
            title="Overview of NFSv4.1 Features">
     <t>
@@ -215,7 +348,8 @@
     </section>
     <section anchor="file system_model" title="File System Model">
       <t>
-        The general file system model used for the NFSv4.1 protocol 
+        The general file system
+        model used for the NFSv4.1 protocol 
         is the same as previous versions.  The server file system is 
         hierarchical with the regular files contained within being 
         treated as opaque byte
@@ -257,8 +391,8 @@
           The NFSv4.1 protocol has a rich and extensible
           attribute structure.  Only a small set of the defined
           attributes are
-          mandatory and must be provided by all server implementations.
-          The other attributes are known as "recommended" attributes.
+          REQUIRED to be provided by all server implementations.
+          The other attributes are known as RECOMMENDED attributes.
         </t>
         <t>
 	  The acl, sacl, and dacl attributes are a significant set of
@@ -376,125 +510,6 @@
       </t>
     </section>
   </section>
-  <section anchor="intro_definitions" title="General Definitions">
-    <t>
-      The following definitions are provided for the purpose of providing
-      an appropriate context for the reader.
-      <list style='hanging'>
-        <t hangText="Byte" anchor="byte">
-          This document defines a byte as an octet, i.e. a datum
-          exactly 8 bits in length.
-        </t>
-        <t hangText="Client" anchor="client_def">
-          The "client" is the entity that accesses the NFS server's
-          resources.  The client may be an application which contains
-          the logic to access the NFS server directly.  The client
-          may also be the traditional operating system client that
-	  provides remote file system services for a set of applications.
-        <vspace blankLines='1' />
-          A client is uniquely identified by a Client Owner.
-        <vspace blankLines='1' />
-          With reference to file locking, the client is also the entity that
-          maintains a set of locks on behalf of one or more
-          applications.  This client is responsible for crash or
-          failure recovery for those locks it manages.
-        <vspace blankLines='1' />
-          Note that multiple clients may share the same transport and
-          connection and
-          multiple clients may exist on the same network node.
-        </t>
-        <t hangText="Client ID">
-          A 64-bit quantity used as a unique, short-hand reference to
-          a client supplied Verifier and client owner.  The server is
-          responsible for supplying the client ID.
-        </t>
-        <t hangText="Client Owner">
-          The client owner is a unique string, opaque to the server,
-          which identifies a client. Multiple network connections and source
-          network addresses originating from those connections may share
-          a client owner. The server is expected to treat requests
-          from connnections with the same client owner as coming from
-          the same client.
-        </t>
-        <t hangText="Lease">
-          An interval of time defined by the server for which the
-          client is irrevocably granted a lock.  At the end of a
-          lease period the lock may be revoked if the lease has not
-          been extended.  The lock must be revoked if a conflicting
-          lock has been granted after the lease interval.
-        <vspace blankLines='1' />
-          All leases granted by a server have the same fixed
-          interval.  Note that the fixed interval was chosen to
-          alleviate the expense a server would have in maintaining
-          state about variable length leases across server failures.
-        </t>
-        <t hangText="Lock">
-          The term "lock" is used to refer to record (byte-range)
-          locks, share reservations, delegations, or layouts unless
-          specifically stated otherwise.
-        </t>
-        <t hangText="Server">
-          The "Server" is the entity responsible for coordinating
-          client access to a set of file systems and is identified by a Server
-          owner. A server can span
-          multiple network addresses.
-        </t>
-        <t hangText="Server Owner">
-          The "Server Owner" identifies the server to the client.
-          The server owner consists of a major and minor identifier.
-          When the client has two connections each to a peer with the
-          same major identifier, the client assumes both peers are
-          the same server (the server namespace is the
-          same via each connection), and assumes and
-          lock state is sharable across both connections. When each peer
-          has both the same major and minor identifier, the client assumes
-          each connection might be associatable with the same session.
-        </t>
-        <t hangText="Stable Storage">
-          NFSv4.1 servers must be able to recover without data
-          loss from multiple power failures (including cascading
-          power failures, that is, several power failures in quick
-          succession), operating system failures, and hardware
-          failure of components other than the storage medium itself
-          (for example, disk, nonvolatile RAM).
-        <vspace blankLines='1' />
-          Some examples of stable storage that are allowable for an
-          NFS server include:
-          <list style='numbers'>
-             <t>
-               Media commit of data, that is, the modified data has
-               been successfully written to the disk media, for
-               example, the disk platter.
-             </t>
-             <t>
-               An immediate reply disk drive with battery-backed on-
-               drive intermediate storage or uninterruptible power
-               system (UPS).
-             </t>
-             <t>
-               Server commit of data with battery-backed intermediate
-               storage and recovery software.
-             </t>
-             <t>
-               Cache commit with uninterruptible power system (UPS) and
-               recovery software.
-             </t>
-           </list>
-        </t>
-        <t hangText="Stateid">
-          A 128-bit quantity returned by a server that uniquely
-          defines the open and locking state provided by the server
-          for a specific open or lock owner for a specific file and
-          type of lock.
-        </t>
-        <t hangText="Verifier">
-          A 64-bit quantity generated by the client that the server
-          can use to determine if the client has restarted and lost
-           all previous lock state.        
-        </t>
-      </list>
-    </t>
-  </section>
   <section anchor="intro_differences" title="Differences from NFSv4.0">
     <t>
       The following summarizes the differences between minor version 
Index: nfsv41_middle_multi_server_ns.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_multi_server_ns.xml,v
retrieving revision 1.31
diff -u -b -w -r1.31 nfsv41_middle_multi_server_ns.xml
--- nfsv41_middle_multi_server_ns.xml	17 Dec 2007 22:17:39 -0000	1.31
+++ nfsv41_middle_multi_server_ns.xml	27 Jan 2008 00:37:36 -0000
@@ -1,10 +1,10 @@
 <!-- 	$Id: nfsv41_middle_multi_server_ns.xml,v 1.31 2007/12/17 22:17:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="multi_server_namespace" title="Multi-Server Namespace">
   <t>
     NFSv4.1 supports attributes that allow a namespace to extend
-    beyond the boundaries of a single server.  It is recommended
+    beyond the boundaries of a single server.  It is RECOMMENDED
     that clients and servers support construction of such
     multi-server namespaces.  Use of such multi-server namespaces 
     is OPTIONAL however, and for many purposes,
@@ -17,7 +17,7 @@
   </t>
   <section anchor="location_attrs" title="Location Attributes">
     <t>
-      NFSv4 contains recommended attributes that allow file systems on
+      NFSv4 contains RECOMMENDED attributes that allow file systems on
       one server to be associated with one or more instances of that
       file system on other servers.  These attributes specify such
       file system instances by specifying a server address
@@ -114,7 +114,7 @@
       as discussed below.
     </t>
     <t>
-      The recommended file system attribute fs_status 
+      The RECOMMENDED file system attribute fs_status 
       can be used to interrogate the present/absent status of a 
       given file system.
     </t>  
@@ -144,11 +144,11 @@
       <t>
         When a GETATTR is done on an absent file system, the set of 
         supported attributes is very limited.  Many attributes, including
-        those that are normally mandatory, will not be available on an
+        those that are normally REQUIRED, will not be available on an
         absent file system.  In addition to the attributes mentioned
         above (fs_locations, fs_locations_info, fs_status), the following
         attributes SHOULD be available on absent file systems, in the
-        case of recommended attributes at least to the same degree that
+        case of RECOMMENDED attributes at least to the same degree that
         they are available on present file systems.
       </t>
       <t>
@@ -195,7 +195,7 @@
         or fs_status, the error NFS4ERR_MOVED will result.  It differs in
         that any appearance in the attribute mask of an attribute not 
         supported for an absent file system (and note that this will
-        include some normally mandatory attributes), will also cause
+        include some normally REQUIRED attributes), will also cause
         an NFS4ERR_MOVED result.
       </t>
     </section>
@@ -239,7 +239,7 @@
         </t>
         <t>
           The unavailability of an attribute because of a file system's
-          absence, even one that is ordinarily mandatory, does not result
+          absence, even one that is ordinarily REQUIRED, does not result
           in any error indication.  The set of attributes returned for
           the root directory of the absent file system in that case is 
           simply restricted to those actually available.
@@ -930,7 +930,7 @@
         multi-vendor transitions they will not.  This poses difficulties, 
         especially for the client without special knowledge  
         of the transition mechanisms adopted by the server.  Note
-        that although fileid is not a mandatory attributes, many servers
+        that although fileid is not a REQUIRED attribute, many servers
         provided them and many clients provide API's that depend on them.
       </t>
       <t>
@@ -2594,7 +2594,7 @@
         substitution.  In case where such variables are intended
         to be used more broadly than a single organization, 
         publication of an informational RFC defining such variables
-        is recommended. 
+        is RECOMMENDED. 
       </t>
       <t>
         The variable ${ietf.org:CPU_ARCH} is used to denote the
Index: nfsv41_middle_op_cb_notify_deviceid.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_cb_notify_deviceid.xml,v
retrieving revision 1.5
diff -u -b -w -r1.5 nfsv41_middle_op_cb_notify_deviceid.xml
--- nfsv41_middle_op_cb_notify_deviceid.xml	22 Dec 2007 18:56:39 -0000	1.5
+++ nfsv41_middle_op_cb_notify_deviceid.xml	27 Jan 2008 00:37:36 -0000
@@ -1,9 +1,9 @@
 <!-- 	$Id: nfsv41_middle_op_cb_notify_deviceid.xml,v 1.5 2007/12/22 18:56:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
-<section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14: CB_NOTIFY_DEVICEID - Notify directory changes" >
+<section anchor="OP_CB_NOTIFY_DEVICEID" title="Operation 14: CB_NOTIFY_DEVICEID - Notify device ID changes" >
   <t>
-    Tell the client of directory changes.
+    Tell the client of deviceID changes.
   </t>

       <list style="hanging">
 
@@ -71,7 +62,7 @@
 	  updated mapping.
 
           The notification is encoded in a value of data
-          type notify_deviceid_chg_or_del4. This data type
+          type notify_deviceid_change4. This data type
           also contains a boolean field, ndc_immediate,
           which if TRUE indicates that the change will be
           enforced immediately, and so the client might not
@@ -82,9 +73,6 @@
           SHOULD get the new device ID to device address
           mappings before issuing new I/O to the device ID.
 
-	  The notification is encoded in a value of
-	  data type notify_deviceid_change4.
-
 	</t>
 
 	<t hangText="NOTIFY4_DEVICEID_DELETE"><vspace />
@@ -112,7 +100,7 @@
           </list>
 
 	  The notification is encoded in a value of data
-	  type notify_deviceid_add_or_delete4.
+	  type notify_deviceid_delete4.
 
           After a server deletes a device ID, it MUST NOT
           reuse that device ID for the same layout type until the
Index: nfsv41_middle_op_getattr.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getattr.xml,v
retrieving revision 1.9
diff -u -b -w -r1.9 nfsv41_middle_op_getattr.xml
--- nfsv41_middle_op_getattr.xml	20 Dec 2007 20:46:10 -0000	1.9
+++ nfsv41_middle_op_getattr.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_getattr.xml,v 1.9 2007/12/20 20:46:10 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_GETATTR" title="Operation 9: GETATTR - Get Attributes" >
 
@@ -41,7 +41,7 @@
       were supported.
     </t>
     <t>
-      All servers must support the mandatory attributes as specified in
+      All servers MUST support the REQUIRED attributes as specified in
       <xref target="absent_getattr">File Attributes</xref>, for all file systems,
       with the exception of absent file systems.
     </t>
Index: nfsv41_middle_op_getdeviceinfo.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdeviceinfo.xml,v
retrieving revision 1.16
diff -u -b -w -r1.16 nfsv41_middle_op_getdeviceinfo.xml
--- nfsv41_middle_op_getdeviceinfo.xml	17 Dec 2007 22:17:39 -0000	1.16
+++ nfsv41_middle_op_getdeviceinfo.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_getdeviceinfo.xml,v 1.16 2007/12/17 22:17:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_GETDEVICEINFO" title="Operation 47: GETDEVICEINFO - Get Device Information" >
 
@@ -20,7 +20,8 @@
 
@@ -34,19 +35,30 @@
 
-
     </t>
     <t>
+      then the device ID's notifications are turned off.

     <t>
      
Index: nfsv41_middle_op_getdevicelist.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_getdevicelist.xml,v
retrieving revision 1.15
diff -u -b -w -r1.15 nfsv41_middle_op_getdevicelist.xml
--- nfsv41_middle_op_getdevicelist.xml	14 Dec 2007 18:17:32 -0000	1.15
+++ nfsv41_middle_op_getdevicelist.xml	27 Jan 2008 00:37:36 -0000
@@ -1,7 +1,7 @@
 <!-- 	$Id: nfsv41_middle_op_getdevicelist.xml,v 1.15 2007/12/14 18:17:32 dnoveck Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
-<section anchor="OP_GETDEVICELIST" title="Operation 48: GETDEVICELIST - Get All Device Mappings" >
+<section anchor="OP_GETDEVICELIST" title="Operation 48: GETDEVICELIST - Get All Device Mappings for a File System" >
 
   <section toc="exclude" anchor="OP_GETDEVICELIST_ARGUMENT" title="ARGUMENT">
    <?rfc include='autogen/getdevicelist_args.xml'?>
@@ -12,82 +12,41 @@
   <section toc="exclude" anchor="OP_GETDEVICELIST_DESCRIPTION" title="DESCRIPTION">
     <t>
       This operation is used by the client to enumerate all of the
-      device ID to device address mappings in use by the server.  An
-      example of the use of this operation is for device types that
-      use SAN devices and in these environments it may be helpful for
-      a client to determine device accessibility upon first file system
-      access.
+      device IDs a server's file system uses.
     </t>
     <t>
-      The client provides the layout type in gdia_layout_type.  Since
+      The client provides a current filehandle of a file object that
+      belongs to the file system (i.e. all file objects sharing the same
+      fsid as that of the current filehandle), and the layout type
+      in gdia_layout_type.  Since
       this operation may require multiple calls to enumerate all the
-      device ID to device address mappings (and is thus
+      device IDs (and is thus
       similar to the <xref target="OP_READDIR">
       READDIR</xref> operation), the client also provides gdia_cookie
       and gdia_cookieverf to specify the current cursor position in the
-      mappings.
-      The client provides gdla_maxcount to limit the number of bytes
-      for the result.  This maximum size represents all of the data
-      being returned within the GETDEVICELIST4resok structure and
-      includes the XDR overhead.  The server may return less data.  If
-      the server is unable to return a single entry device list item
-      within the gdla_maxcount limit, the error NFS4ERR_TOOSMALL will
-      be returned.
+      list.
+      The client provides gdla_maxdevices to limit the number of device IDs
+      in the result.
+      The server MAY return fewer device IDs.
     </t>
     
     <t>
   <section toc="exclude" anchor="OP_GETDEVICELIST_IMPLEMENTATION" title="IMPLEMENTATION">
+      An example of the use of this operation is for pNFS
+      clients and servers that use LAYOUT4_BLOCK_VOLUME
+      layouts.  In these environments it may be helpful
+      for a client to determine device accessibility upon
+      first file system access.
 
   </t>
-  <t>
-   See <xref target="OP_GETDEVICEINFO_IMPLEMENTATION"/> for a discussion
-   of races between GETDEVICELIST and CB_NOTIFY_DEVICEID.
-  </t>
-
   </section>
 </section>
Index: nfsv41_middle_op_layoutcommit.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_layoutcommit.xml,v
retrieving revision 1.20
diff -u -b -w -r1.20 nfsv41_middle_op_layoutcommit.xml
--- nfsv41_middle_op_layoutcommit.xml	17 Dec 2007 22:17:39 -0000	1.20
+++ nfsv41_middle_op_layoutcommit.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_layoutcommit.xml,v 1.20 2007/12/17 22:17:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_LAYOUTCOMMIT" title="Operation 49: LAYOUTCOMMIT - Commit writes made using a layout" >
 
@@ -124,7 +124,7 @@
   </section>
   <section toc="exclude" anchor="OP_LAYOUTCOMMIT_IMPLEMENTATION" title="IMPLEMENTATION">
     <t>
-      Optionally, the client can also use LAYOUTCOMMIT with the
+      The client MAY also use LAYOUTCOMMIT with the
       loca_reclaim field set to TRUE to convey hints to modified file
       attributes or to report layout-type specific information such as
       I/O errors for object-based storage layouts, as normally done
Index: nfsv41_middle_op_mandlist.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_mandlist.xml,v
retrieving revision 1.4
diff -u -b -w -r1.4 nfsv41_middle_op_mandlist.xml
--- nfsv41_middle_op_mandlist.xml	16 Dec 2007 21:07:51 -0000	1.4
+++ nfsv41_middle_op_mandlist.xml	27 Jan 2008 00:37:36 -0000
@@ -1,17 +1,20 @@
+<!-- 	$Id: $	 -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <section anchor="operation_mandlist" 
 	 title="Operations: REQUIRED, RECOMMENDED, or OPTIONAL">
 
   <t>
     The following tables summarize the operations of the NFSv4.1
-    protocol and the corresponding designation of mandatory, optional
-    or mandatory not to implement.  The designation of mandatory not
+    protocol and the corresponding designation of REQUIRED, RECOMMENDED,
+    OPTIONAL
+    or MUST NOT to implement.  The designation of MUST NOT
     to implement is reserved for those operations that were defined in
     NFSv4.0 and they MUST NOT be implemented in NFSv4.1.  These
     operations are limited to those replaced by the Sessions
     functionality of NFSv4.1.
   </t>
   <t>
-    For the most part, the mandatory or optional designation is for
+    For the most part, the REQUIRED, RECOMMENDED, or OPTIONAL designation is for
     the server implementation.  The client is generally required to
     implement the operations needed for the operating environment for
     which it serves.  For example, a read-only NFSv4.1 client would
@@ -44,17 +47,17 @@
     </list>
   </t>
 
-  <t> For the NFSv4.1 features that are optional, the operations that
-    support those features are optional and the server would return
+  <t> For the NFSv4.1 features that are OPTIONAL, the operations that
+    support those features are OPTIONAL and the server would return
     NFS4ERR_NOTSUPP in response to the client's use of those
-    operations.  If a optional feature is supported, it is possible
-    that a set of operations related to the feature become mandatory
+    operations.  If an OPTIONAL feature is supported, it is possible
+    that a set of operations related to the feature become REQUIRED
     to implement.  The third column of the table designates the
-    feature(s) and if the operation is mandatory or optional in the
+    feature(s) and if the operation is REQUIRED or OPTIONAL in the
     presence of support for the feature.
   </t>
   <t>
-    The optional features identified and their abbreviations are as
+    The OPTIONAL features identified and their abbreviations are as
     follows:
     <list style="hanging">
       <t hangText="pNFS">
Index: nfsv41_middle_op_nverify.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_nverify.xml,v
retrieving revision 1.6
diff -u -b -w -r1.6 nfsv41_middle_op_nverify.xml
--- nfsv41_middle_op_nverify.xml	8 Nov 2007 04:15:21 -0000	1.6
+++ nfsv41_middle_op_nverify.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_nverify.xml,v 1.6 2007/11/08 04:15:21 shepler Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_NVERIFY" title="Operation 17: NVERIFY - Verify Difference in Attributes" >
 
@@ -39,7 +39,7 @@
       </figure>
     </t>
     <t>
-      In the case that a recommended attribute is specified in the NVERIFY
+      In the case that a RECOMMENDED attribute is specified in the NVERIFY
       operation and the server does not support that attribute for the
       file system object, the error NFS4ERR_ATTRNOTSUPP is returned to the
       client.
Index: nfsv41_middle_op_secinfo_no_name.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_secinfo_no_name.xml,v
retrieving revision 1.12
diff -u -b -w -r1.12 nfsv41_middle_op_secinfo_no_name.xml
--- nfsv41_middle_op_secinfo_no_name.xml	12 Nov 2007 03:20:23 -0000	1.12
+++ nfsv41_middle_op_secinfo_no_name.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_secinfo_no_name.xml,v 1.12 2007/11/12 03:20:23 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="Operation 52: SECINFO_NO_NAME - Get Security on Unnamed Object" anchor="OP_SECINFO_NO_NAME">
   <t>
@@ -40,7 +40,7 @@
       is, because there is no way to query.  Therefore, the client
       must iterate through the security triples available at the
       client and reattempt the PUTFH, PUTROOTFH or PUTPUBFH operation.
-      In the unfortunate event none of the MANDATORY security triples
+      In the unfortunate event none of the REQUIRED security triples
       are supported by the client and server, the client SHOULD try
       using others that support integrity. Failing that, the client
       can try using other forms (e.g. AUTH_SYS and AUTH_NONE), but
Index: nfsv41_middle_op_verify.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_op_verify.xml,v
retrieving revision 1.6
diff -u -b -w -r1.6 nfsv41_middle_op_verify.xml
--- nfsv41_middle_op_verify.xml	8 Nov 2007 04:15:21 -0000	1.6
+++ nfsv41_middle_op_verify.xml	27 Jan 2008 00:37:36 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_op_verify.xml,v 1.6 2007/11/08 04:15:21 shepler Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="OP_VERIFY" title="Operation 37: VERIFY - Verify Same Attributes" >
 
@@ -40,7 +40,7 @@
       avoid the unintended result.
     </t>
     <t>
-      In the case that a recommended attribute is specified in the VERIFY
+      In the case that a RECOMMENDED attribute is specified in the VERIFY
       operation and the server does not support that attribute for the
       file system object, the error NFS4ERR_ATTRNOTSUPP is returned to the
       client.
Index: nfsv41_middle_pnfs.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_pnfs.xml,v
retrieving revision 1.68
diff -u -b -w -r1.68 nfsv41_middle_pnfs.xml
--- nfsv41_middle_pnfs.xml	16 Jan 2008 05:24:14 -0000	1.68
+++ nfsv41_middle_pnfs.xml	27 Jan 2008 00:37:37 -0000
@@ -1,10 +1,10 @@
 <!-- $Id: nfsv41_middle_pnfs.xml,v 1.68 2008/01/16 05:24:14 mre Exp $ -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="Parallel NFS (pNFS)">
 <section title="Introduction" anchor="pnfs_intro">
 <t>
-  pNFS is a set of optional features within NFSv4.1; the pNFS feature
+  pNFS is an OPTIONAL feature within NFSv4.1; the pNFS feature
   set allows direct client access to the storage devices containing
   file data.  When file data for a single NFSv4 server is stored on
   multiple and/or higher throughput storage devices (by comparison to
@@ -170,7 +170,7 @@
   required by the storage devices to perform client access control.
 </t>
 <t>
-  A particular control protocol is not mandated by NFSv4.1 but
+  A particular control protocol is not REQUIRED by NFSv4.1 but
   requirements are placed on the control protocol for maintaining
   attributes like modify time, the change attribute, and the
   end-of-file (EOF) position.
@@ -371,8 +371,8 @@
  </t>
  <t hangText="GETDEVICELIST">
   (<xref target="OP_GETDEVICELIST" />),
-  allows clients to fetch all of the mappings of device IDs to storage
-  device addresses for a specific file system.
+  allows clients to fetch all device IDs
+  for a specific file system.
  </t>
  <t hangText="LAYOUTGET">
   (<xref target="OP_LAYOUTGET" />) is used by a client to get
@@ -1283,7 +1283,7 @@
   If a file is to be created on a pNFS enabled file
   system, the client uses the OPEN operation.  With the
   normal set of attributes that may be provided upon OPEN
-  used for creation, there is an optional layout_hint
+  used for creation, there is an OPTIONAL layout_hint
   attribute.  The client's use of layout_hint allows the
   client to express its preference for a layout type and its
   associated layout details. The use a createmode4 of
@@ -1323,8 +1323,7 @@
 <t>
   When the client wants to send an I/O, it determines which device ID
   it needs to send the I/O command to by examining the data
-  description in the layout. It then sends a GETDEVICELIST to return
-  a list of all device ID to device address mappings, or a
+  description in the layout. It then sends a
   GETDEVICEINFO to find the device address(es) of the device ID.  The
   client then sends the I/O request one of device ID's device addresses, using the
   storage protocol defined for the layout type.
Index: nfsv41_middle_security_considerations.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_security_considerations.xml,v
retrieving revision 1.9
diff -u -b -w -r1.9 nfsv41_middle_security_considerations.xml
--- nfsv41_middle_security_considerations.xml	17 Dec 2007 22:17:39 -0000	1.9
+++ nfsv41_middle_security_considerations.xml	27 Jan 2008 00:37:37 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_security_considerations.xml,v 1.9 2007/12/17 22:17:39 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section anchor="securityconsider" title="Security Considerations">
 <t>
@@ -17,8 +17,8 @@
    came from the same network address and port number
    that the request was sent to.  While such a model is
    easy to implement and simple to deploy and use, it is
-   certainly not a safe model.  Thus, NFSv4.1 mandates that
-   implementations support a security model that uses end
+   certainly not a safe model.  Thus, NFSv4.1 
+   implementations are REQUIRED to support a security model that uses end
    to end authentication, where an end-user on a client
    mutually authenticates (via cryptographic schemes that
    do not expose passwords or keys in the clear on the
@@ -35,7 +35,7 @@
    machine authentication via network address checking
    and AUTH_SYS identification can still be supported
    with the caveat that the AUTH_SYS flavor is neither
-   MANDATORY nor RECOMMENDED by this specification, and
+   REQUIRED nor RECOMMENDED by this specification, and
    so interoperability via AUTH_SYS is not assured.
 
 </t>
Index: nfsv41_middle_state_mgmt.xml
===================================================================
RCS file: /cvs/stdk/NFSv41/nfsv41_middle_state_mgmt.xml,v
retrieving revision 1.22
diff -u -b -w -r1.22 nfsv41_middle_state_mgmt.xml
--- nfsv41_middle_state_mgmt.xml	16 Jan 2008 04:57:37 -0000	1.22
+++ nfsv41_middle_state_mgmt.xml	27 Jan 2008 00:37:37 -0000
@@ -1,5 +1,5 @@
 <!-- 	$Id: nfsv41_middle_state_mgmt.xml,v 1.22 2008/01/16 04:57:37 mre Exp $	 -->
-<!-- Copyright (C) The IETF Trust (2007) -->
+<!-- Copyright (C) The IETF Trust (2007-2008) -->
 <!-- Copyright (C) The Internet Society (2006) -->
 <section title="State Management" >
   <t>
@@ -1353,8 +1353,8 @@
       separate from locking itself.
     </t>
     <t>
-      The following operations have become mandatory-to-not-implement.
-      The server should return NFS4ERR_NOTSUPP if these operations are
+      The following NFSv4.0 operations MUST NOT be implemented in NFSv4.1.
+      The server MUST return NFS4ERR_NOTSUPP if these operations are
       found in an NFSv4.1 COMPOUND.
       <list style='symbols'>
         <t>
Index: dotx-id.d/Makefile
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx-id.d/Makefile,v
retrieving revision 1.11
diff -u -b -w -r1.11 Makefile
--- dotx-id.d/Makefile	31 Dec 2007 22:51:37 -0000	1.11
+++ dotx-id.d/Makefile	27 Jan 2008 00:37:37 -0000
@@ -1,4 +1,4 @@
-#
+# Copyright (C) The IETF Trust (2007)
 # Manage the .xml for the NFSv4 minor version document.
 #
 #	$Id: Makefile,v 1.11 2007/12/31 22:51:37 mre Exp $
Index: dotx.d/Makefile
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/Makefile,v
retrieving revision 1.43
diff -u -b -w -r1.43 Makefile
--- dotx.d/Makefile	10 Jan 2008 16:26:11 -0000	1.43
+++ dotx.d/Makefile	27 Jan 2008 00:37:37 -0000
@@ -1,3 +1,4 @@
+# Copyright (C) The IETF Trust (2007-2008)
 # $Id: Makefile,v 1.43 2008/01/10 16:26:11 mre Exp $
 
 FORE_OPS_AS_IS1 = \
@@ -166,7 +167,6 @@
+++ dotx.d/cb_notify_args.x	27 Jan 2008 00:37:37 -0000
 
 struct notify4 {
+++ dotx.d/cb_notify_deviceid_args.x	27 Jan 2008 00:37:37 -0000

+++ dotx.d/get_dir_delegation_args.x	27 Jan 2008 00:37:37 -0000

+++ dotx.d/get_dir_delegation_res.x	27 Jan 2008 00:37:37 -0000
+++ dotx.d/getdeviceinfo_args.x	27 Jan 2008 00:37:37 -0000
 
+++ dotx.d/getdeviceinfo_res.x	27 Jan 2008 00:37:37 -0000
 
+++ dotx.d/getdevicelist_args.x	27 Jan 2008 00:37:37 -0000
 
+++ dotx.d/getdevicelist_res.x	27 Jan 2008 00:37:37 -0000
 
 union GETDEVICELIST4res switch (nfsstat4 gdlr_status) {
 case NFS4_OK:
 	GETDEVICELIST4resok	gdlr_resok4;
-case NFS4ERR_TOOSMALL:
-	count4 gdlr_mincount;
 default:
 	void;
 };
Index: dotx.d/head.x
===================================================================
RCS file: /cvs/stdk/NFSv41/dotx.d/head.x,v
retrieving revision 1.53
diff -u -b -w -r1.53 head.x
--- dotx.d/head.x	16 Dec 2007 21:07:55 -0000	1.53
+++ dotx.d/head.x	27 Jan 2008 00:37:37 -0000
@@ -1,5 +1,5 @@
 /*
- *  Copyright (C) The IETF Trust (2007)
+ *  Copyright (C) The IETF Trust (2007-2008)
  *  All Rights Reserved.
  *
  *  Copyright (C) The Internet Society (1998-2006).
@@ -355,8 +355,6 @@
 include(type_deviceid4.x)
 include(type_device_addr4.x)
 
+++ dotx.d/spit_types.sh	27 Jan 2008 00:37:37 -0000
@@ -457,16 +457,6 @@
 EOF
         ;;
 
-	type_devlist_item4.x )
-
-cat << EOF > $i
-struct devlist_item4 {
-        deviceid4                 dli_id;
 
 cat << EOF > $i

---

From: "J. Bruce Fields" <[email-masked]>
Date: Sat, 26 Jan 2008 22:43:28 -0500

>  first file system access.

OK!

> attributes chapter and other chapters. Goodbye MANDATORY (you never

Yowza.  Thanks for doing all this.

The "and that" isn't right.  Maybe "which requires that"?  Or "after
which"?:

	invokes EXCHANGE_ID with the user's credential, specifying
	SP4_MACH_CRED, after which all operations MUST be protected with
	the machine credential.

>       use the same credential that created the client ID.

So maybe just avoid the redundancy and just go with the second
sentence?:

	invokes EXCHANGE_ID with the user's credential, specifying
	SP4_MACH_CRED.  The server will then reject any subsequent
	operations on the client ID or its sessions that do not use
	RPCSEC_GSS with privacy or integrity and do not use the same
	credential that created the client ID.

(Unless I'm missing some subtlety here.)

> Index: nfsv41_middle_fileattributes.xml

I'm of two minds.

It may be true that "mandatory attributes" are attributes that servers
are REQUIRED to implement, but that doesn't mean we need to rename
"mandatory attributes" to "REQUIRED attributes".  All we need to do is
state the requirement once in the definition of "mandatory attributes".

I understand the desire to include the IETF terms in the names, but
there's also an advantage to terminology that is already established
(hence already sprinkled around mailing list archives and code comments,
not to mention 3530).

> Index: nfsv41_middle_fileattributes_acls.xml

The purpose of this sentence is to make it as clear as possible that the
following MUST's apply only to servers that implement certain ACL
attributes already.  So it's the negative ("optional") that needs
calling attention to, not the positive ("RECOMMENDED").  Otherwise the
following "However," for example, no longer makes sense.

So I'd prefer that change be reverted.  But if it's important to restate
that these attributes are RECOMMENDED, we could make it:

	Support for the ACL attributes is RECOMMENDED, not REQUIRED.
	However, a server that supports ...

> +            if it supports AUDIT or ALARM ACEs, then it MUST support

Nothing else jumps out at me.

--b.

---

From: Benny Halevy <[email-masked]>
Date: Sun, 27 Jan 2008 09:02:11 +0200

Mike, in addition to Bruce's comments, I spotted the following nit:

"MUST NOT to implement" doesn't sound right.  How about this:

     The following tables summarize the operations of the NFSv4.1
-    protocol and the corresponding designation of mandatory, optional
-    or mandatory not to implement.  The designation of mandatory not
-    to implement is reserved for those operations that were defined in
-    NFSv4.0 and they MUST NOT be implemented in NFSv4.1.  These
-    operations are limited to those replaced by the Sessions
+    protocol and the corresponding designation of MUST NOT, OPTIONAL,
+    RECOMMENDED, or REQUIRED to implement.  The designation of MUST NOT
+    implement is reserved for those operations that were defined in
+    NFSv4.0;  these
+    are limited to those replaced by the Sessions
     functionality of NFSv4.1.

Benny