Subject: Re: [nfsv4] New LAYOUTCOMMIT text for v4.2

From: "Haynes, Tom" <[email-masked]>
Date: Tue, 27 Dec 2011 16:11:10 +0000

This is a lot of text for a small change.

11.9.2 illustrates an important editorial point:

   The ARGUMENT specification of the LAYOUTCOMMIT operation in section
   18.42.1 of [2] is unchanged, but note that the following two
   arguments have no functionality - they SHOULD be set to zero by the
   client and MUST be ignored by the server:

           offset4                 loca_offset;
           length4                 loca_length;

This is also stated in the errata. Once it appears as errata, it is part of the document.

The intent of the NFSv4,2 document is not to restate what is in 5661 - that will be the
job for 5661bis.

What is new here? What changes are just for NFSv4.2?

On Nov 27, 2011, at 6:55 PM, <[email-masked]>
 <[email-masked]> wrote:

> Benny's comments on the 4.1 errata text also affect the 4.2 (minorversion2) text.  New version (v2) is attached.
> 
> Thanks,
> --David
> 
>> -----Original Message-----
>> From: Black, David
>> Sent: Saturday, November 26, 2011 8:25 PM
>> To: Black, David; [email-masked]
>> Cc: [email-masked]; [email-masked]
>> Subject: [nfsv4] New LAYOUTCOMMIT text for v4.2
>> 
>> And here's an initial draft of text for the minorversion2 draft.
>> 
>> Thanks,
>> --David
>> 
>> 
>>> -----Original Message-----
>>> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
>>> Sent: Saturday, November 26, 2011 7:57 PM
>>> To: [email-masked]
>>> Cc: [email-masked]; [email-masked]
>>> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
>>> 
>>> What I think is the final errata text for this change to RFC 5661 is attached - sorry it took so
>>> long to get back to this.  In the absence of objection, I'll file the errata request.
>>> 
>>> The next item is to do something for NFSv4.2 - I will go write up some change text for the
>>> minorversion2 draft along the lines of what was done for the LAYOUTRETURN changes there.
>>> I don't plan to change the arguments to LAYOUTCOMMIT.
>>> 
>>> Thanks,
>>> --David
>>> 
>>>> -----Original Message-----
>>>> From: Benny Halevy [mailto:[email-masked]]
>>>> Sent: Monday, July 11, 2011 1:58 AM
>>>> To: Black, David
>>>> Cc: [email-masked]; [email-masked]
>>>> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
>>>> 
>>>> On 2011-07-11 04:47, [email-masked] wrote:
>>>>> My rationale for the client to put EOF in loca_length was to get reasonable
>>>>> behavior if a new client happens to talk to an old server that hasn't implemented
>>>>> the "MUST ignore" requirement.
>>>>> 
>>>>> If loca_length is set to EOF (or some client view of what EOF is) by the client,
>>>>> this new-client-old-server scenario only causes issues for the pNFS block layout.
>>>>> If loca_length is set to 0 by the client, then this is also a concern for the
>>>>> file and object layouts.
>>>>> 
>>>>> A subtlety here is that this is about errata for the existing NFSv4.1 RFC (5661).
>>>>> I have no problem with NFS4.2 using "MUST" for everything (client MUST set both
>>>>> values to 0, server MUST ignore them), based on the visible bump in the minor
>>>>> version number.
>>>>> 
>>>>> How confident are people that the "MUST ignore" requirement for the byte range
>>>>> will get into all the 4.1 servers so that this new-client-old-server scenario
>>>>> can't happen?
>>>> 
>>>> We either commit to this change or not.
>>>> A requirement for backward compatibility means server SHOULD ignore rather than
>>>> MUST ignore, which we don't want to do.
>>>> 
>>>> Since there are no commercial pnfs clients nor servers out in the market I'm
>>>> not worried about server vendors that do not follow the errata.
>>>> I expect all interested vendors to follow the spec + errata and the working
>>>> group discussion (as well as participate in the testing events, but this is
>>>> a separate issue).
>>>> 
>>>> Benny
>>>> 
>>>>> 
>>>>> Thanks,
>>>>> --David
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Benny Halevy [mailto:[email-masked]]
>>>>>> Sent: Sunday, July 10, 2011 7:13 AM
>>>>>> To: Black, David
>>>>>> Cc: Welch, Brent; Mike Eisler; [email-masked]
>>>>>> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
>>>>>> 
>>>>>> On 2011-07-09 20:53, Welch, Brent wrote:
>>>>>>> Yeah, LAYOUTCOMMIT already takes loca_last_write_offset which is all the client knows.
>>>> loca_length
>>>>>> should be 0.
>>>>>> 
>>>>>> Agreed.
>>>>>> 
>>>>>> Benny
>>>>>> 
>>>>>>> 	Brent
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Mike Eisler
>>>>>>> Sent: Saturday, July 09, 2011 7:53 AM
>>>>>>> To: [email-masked]
>>>>>>> Subject: Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
>>>>>>> 
>>>>>>> Apologies if there is a discussion I've missed, but:
>>>>>>> 
>>>>>>>   The
>>>>>>>   client SHOULD set loca_offset to 0 and SHOULD set loca_length to the
>>>>>>>   offset that the client expects for EOF after the LAYOUTCOMMIT.
>>>>>>>   The server MUST ignore the loca_offset and loca_length arguments.
>>>>>>> 
>>>>>>> If the server MUST ignore loca_length, then why burden the client with a
>>>>>>> mandate to determine the "offset that the client expects for EOF after the
>>>>>>> LAYOUTCOMMIT."
>>>>>>> 
>>>>>>> And BTW, I'm not sure what "offset that the client expects for EOF after
>>>>>>> the LAYOUTCOMMIT" is.
>>>>>>> 
>>>>>>> Regardless, I suggest,
>>>>>>> 
>>>>>>>   The
>>>>>>>   client SHOULD set both loca_offset and loca_length to 0.
>>>>>>> 
>>>>>>> 
>>>>>>> On Fri, July 8, 2011 5:39 pm, [email-masked] wrote:
>>>>>>>> A day late, but here's the revised errata text for this that includes
>>>>>>>> everything Benny
>>>>>>>> found.  See attached file.
>>>>>>>> 
>>>>>>>> I've tried to deal with the variation of commit effect among layout types
>>>>>>>> (file and
>>>>>>>> object commit the entire layout, block doesn't) by using a concept of the
>>>>>>>> "scope" of
>>>>>>>> a LAYOUTCOMMIT operation to replace references to the byte range defined
>>>>>>>> by the
>>>>>>>> to-be-deprecated loca_offset and loca_length arguments.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> --David
>>>>>>>> ----------------------------------------------------
>>>>>>>> David L. Black, Distinguished Engineer
>>>>>>>> EMC Corporation, 176 South St., Hopkinton, MA  01748
>>>>>>>> +1 (508) 293-7953             FAX: +1 (508) 293-7786
>>>>>>>> [email-masked]        Mobile: +1 (978) 394-7754
>>>>>>>> ----------------------------------------------------
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> nfsv4 mailing list
>>>>>>>> [email-masked]
>>>>>>>> https://www.ietf.org/mailman/listinfo/nfsv4
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> nfsv4 mailing list
>>>>> [email-masked]
>>>>> https://www.ietf.org/mailman/listinfo/nfsv4
> 
> <LAYOUTCOMMIT byte range deprecate 4-2 v2.txt>_______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: <[email-masked]>
Date: Tue, 27 Dec 2011 14:53:24 -0500

Tom,

> This is a lot of text for a small change.

[... snip ...]

> This is also stated in the errata. Once it appears as errata, it is part of the document.

Only if it's approved - we'll have to see what happens.  The size of the changes
required suggests "Hold for Document Update" as a likely result as opposed to
"Approved" - I'll go submit the errata to start the ball rolling.

Independent of that, I believe that this should be published in an RFC, not just left
for people to go find as errata, and the 4.2 draft is the next opportunity to do so.

Thanks,
--David

> From: Haynes, Tom [mailto:[email-masked]]
> Sent: Tuesday, December 27, 2011 11:11 AM
> To: Black, David
> Cc: <[email-masked]>
> Subject: Re: [nfsv4] New LAYOUTCOMMIT text for v4.2
>

---

From: <[email-masked]>
Date: Tue, 27 Dec 2011 15:18:43 -0500

All 5 errata for this change have just been submitted - I don't know who "wins" the honor (prize?) of processing them, but enjoy ;-).

Thanks,
--David

> Sent: Monday, November 28, 2011 6:41 AM
> To: Black, David
> Cc: [email-masked]
> Thanks.
> Looks good!
> On 2011-11-28 02:52, [email-masked] wrote:
> > Hi Benny,
> >
> >
> >
> > I've corrected the nit and add the sentence that you suggested for layout coverage.
> >
> >
> >
> > On this question:
> >
> >
> >
> >> Out of curiosity, how does the client tell the server about a new loca_last_write_offset
> >
> >> when writing in-place?
> >
> >> Can it send LAYOUTCOMMIT for extents already in the PNFS_BLOCK_READ_WRITE_DATA
> >
> >> state? (It seems to be allowed by RFC5663)
> >
> >
> >
> > It's allowed, and a check w/our implementers suggests that "yes" is a good answer.
> >
> > I've deleted the offending sentence.
> >
> >
> >
> > Revised (v3) text is attached.
> >
> >
> >
> > Thanks,
> > --David
> >
> >
> >
> > *From:*Benny Halevy [mailto:[email-masked]]
> > *Sent:* Sunday, November 27, 2011 5:32 AM
> > *To:* Black, David
> > *Cc:* [email-masked]; [email-masked]
> > *Subject:* Re: [nfsv4] New LAYOUTCOMMIT errata text for 5661
> >
> >
> >
> > Thanks for getting back to this.
> >
> > Overall the proposed changes look good to me.
> >
> > I have a few comments listed below:
> >
> >
> >
> >>   The LAYOUTCOMMIT operation commits changes in the layout represented
> >
> >>   by the current filehandle, client ID (derived from the session ID in
> >
> >>   the preceding SEQUENCE operation), and stateid.  As a layout-independent
> >
> >>   operation, LAYOUTCOMMIT commits the entire layout; layout-specific data
> >
> >
> >
> > nit: it's more correct to define this as "layout type-specific" rather than "layout-specific"
> >
> >
> >
> >>   (loca_layoutupdate) may specify a smaller scope of data that is to
> >
> >>   be committed (e.g., for the block layout, see RFC 5663 [41]).
> >
> >
> >
> >
> >
> >>   The loca_offset and loca_length arguments have been deprecated.  The
> >
> >>   The client MUST hold one or more existing layouts
> >
> >>   previously granted via LAYOUTGET (Section 18.43), with an iomode of
> >
> >>   LAYOUTIOMODE4_RW.
> >
> >
> >
> > In case layout type-specific data restrict the scope of the LAYOUTCOMMIT
> >
> > operation, the client MUST hold one or more existing layouts with an iomode
> >
> > of LAYOUTIOMODE4_RW fully covering the committed byte ranges.
> >
> >
> >
> >>   In the case where no previously granted layout
> >
> >>   has an iomode of LAYOUTIOMODE4_RW, the server should return the error
> >
> >>   NFS4ERR_BAD_IOMODE.  For the case where the client does not hold
> >
> >>   any previously granted layout, the server should return the error
> >
> >>   NFS4ERR_BAD_LAYOUT.
> >
> >
> >
> >>   The loca_last_write_offset field specifies the offset of the last
> >
> >>   byte written by the client previous to the LAYOUTCOMMIT.  Note that
> >
> >>   this value is never equal to the file's size (at most it is one byte
> >
> >>   less than the file's size) and MUST be less than or equal to
> >
> >>   NFS4_MAXFILEOFF.  Also, loca_last_write_offset MUST be within the
> >
> >>   scope of data committed by the LAYOUTCOMMIT.
> >
> >
> >
> > Out of curiosity, how does the client tell the server about a new loca_last_write_offset
> >
> > when writing in-place?
> >
> > Can it send LAYOUTCOMMIT for extents already in the PNFS_BLOCK_READ_WRITE_DATA
> >
> > state? (It seems to be allowed by RFC5663)
> >
> >
> >
> > Benny
> >
> >
> >
> > On Sun, Nov 27, 2011 at 2:56 AM, <[email-masked] <mailto:[email-masked]>> wrote:
> >
> >> From: Benny Halevy [mailto:[email-masked] <mailto:[email-masked]>]
> >
> >> Cc: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> >> On 2011-07-11 04:47, [email-masked] <mailto:[email-masked]> wrote:
> >> >> From: Benny Halevy [mailto:[email-masked] <mailto:[email-masked]>]
> >> >> Cc: Welch, Brent; Mike Eisler; [email-masked] <mailto:[email-masked]>
> >> >>> From: [email-masked] <mailto:[email-masked]> [mailto:[email-masked]
> <mailto:[email-masked]>] On Behalf Of Mike Eisler
> >> >>> To: [email-masked] <mailto:[email-masked]>
> >> >>> On Fri, July 8, 2011 5:39 pm, [email-masked] <mailto:[email-masked]> wrote:
> >> >>>> +1 (508) 293-7953 <tel:%2B1%20%28508%29%20293-7953>             FAX: +1 (508) 293-7786
> <tel:%2B1%20%28508%29%20293-7786>
> >> >>>> [email-masked] <mailto:[email-masked]>        Mobile: +1 (978) 394-7754
> <tel:%2B1%20%28978%29%20394-7754>
> >> >>>> [email-masked] <mailto:[email-masked]>
> >> > [email-masked] <mailto:[email-masked]>

---

From: "Haynes, Tom" <[email-masked]>
Date: Wed, 28 Dec 2011 18:12:09 +0000

> "Approved" - I'll go submit the errata to start the ball rolling.

I think we have shown our support for these changes as a WG - both on the mailing list and at the various
IETF meetings.

> for people to go find as errata, and the 4.2 draft is the next opportunity to do so.

They describe a set of changes to NFSv4.1 and not NFSv4.2. An implementor trying to build a compliant
NFSv4.1 offering would expect to only read the NFSv4.1 spec and not the NFSv4.2 spec.

If we had set up our minor versioning rules differently, I would agree with you on placing it in NFSv4.2.

However, I happen to agree that chasing down errata is the same basic issue - we don't want information
in two places. The other option in my mind is 5661bis, but I think that process is too heavy for what we
want.