Subject: [nfsv4] New version of sparse draft (draft-hildebrand-nfsv4-read-sparse-01.txt)

From: Dean Hildebrand <[email-masked]>
Date: Wed, 29 Sep 2010 16:41:09 -0700

Hello,

I uploaded a new version of our internet draft "Simple and Efficient 
Read Support for Sparse Files".

http://www.ietf.org/id/draft-hildebrand-nfsv4-read-sparse-01.txt

Please have a look and give us any feedback.

The main changes are:
1) Added section regarding pNFS (which basically states that there are 
no real changes to how pNFS works)
2) Added example sparse read message flow
3) Added statement to the effect that if a client sends a read request 
for a chunk that ends in zeroes, the server can return a short read 
(thanks to Benny)
4) I didn't change the return value.  There was a suggestion to use a 
flag instead of overloaded the length parameter, but I'm not sure what 
this gives us (although it might be more clear in the doc)

Dean

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 30 Sep 2010 13:59:27 -0400

On Wed, Sep 29, 2010 at 04:41:09PM -0700, Dean Hildebrand wrote:
> 1) Added section regarding pNFS (which basically states that there
> are no real changes to how pNFS works)
> 3) Added statement to the effect that if a client sends a read
> request for a chunk that ends in zeroes, the server can return a
> short read (thanks to Benny)
> 4) I didn't change the return value.  There was a suggestion to use
> a flag instead of overloaded the length parameter, but I'm not sure
> what this gives us (although it might be more clear in the doc)

How will the client take advantage of the data_length field?

--b.

---

From: Dean Hildebrand <[email-masked]>
Date: Thu, 30 Sep 2010 11:22:34 -0700

> How will the client take advantage of the data_length field?
I'm not 100% sure what you are asking, but all read requests into the 
region specified by data_offset and data_length cannot be assumed to be 
zero.  The data must come from either the client cache or server.  The 
example was designed to answer this type of question, does it help?
Dean

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 30 Sep 2010 14:37:52 -0400

On Thu, Sep 30, 2010 at 11:22:34AM -0700, Dean Hildebrand wrote:
> I'm not 100% sure what you are asking, but all read requests into
> the region specified by data_offset and data_length cannot be
> assumed to be zero.  The data must come from either the client cache
> or server.  The example was designed to answer this type of
> question, does it help?

I think I understand what data_offset *means*.  What I don't understand
is how the client can use data_offset to improve performance.

--b.

---

From: Benny Halevy <[email-masked]>
Date: Thu, 30 Sep 2010 21:12:11 +0200

On 2010-09-30 11:15, Benny Halevy wrote:
> On 2010-09-30 01:41, Dean Hildebrand wrote:
>>  Hello,
>>
>> I uploaded a new version of our internet draft "Simple and Efficient Read Support for Sparse Files".
>>
>> 1) Added section regarding pNFS (which basically states that there are no real changes to how pNFS works)
>> 3) Added statement to the effect that if a client sends a read request for a chunk that ends in zeroes, the server can return a short read (thanks to Benny)
>> 4) I didn't change the return value.  There was a suggestion to use a flag instead of overloaded the length parameter, but I'm not sure what this gives us (although it might be more clear in the doc)
>>
>> Dean
>>
>>
>>
>> _______________________________________________
>> nfsv4 mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/nfsv4
> 
> Dean, a few comments:
> 
> 1) Section 5.2:
>       "If data_length is not zero, then the data in the
>       file from data_offset until data_length is allocated"
> 
> should be:
>       "If data_length is not zero, then the data in the
>       file from data_offset until data_offset+data_length is allocated"
> 
> 2)    Either way, the client can ignore
>       the information in READ4reshole.
> 
> 	This sounds too weak.  Either the metadata the server return is useful or not.
> 
> 	When data_length is not zero, the server MUST provide the same semantics
> 	for READ4reshole as if the client read the region and received zeroes;
> 	the implied holes contents lifetime MUST be exactly the same as any other read data.

Sorry, the strong semantics are desired for the range up until data_offset,
not <data_offset, data_offset - data_length>

For example, say the client sent a READ for offset 0, count 0x1000
and the server returned NFS4ERR_HOLE with data_offset 0x4000, and some data_length.

This means that the area between 0x1000 and 0x4000 is a hole.

Benny

> 
> 	The client MAY use the information in READ4reshole to optimize access
> 	to the server and avoid sending READ requests for the region.
> 
> 3) Section 5.3:
>   "when a data server is returning a
>    READ4reshole structure, it should still contain the offset and length
>    of the next allocated block in the file, even if that block is not
>    located on that particular data server."
> 
> 	First, "should" is problematic.  Assuming you meant "SHOULD", what
> 	would the DS return in case it doesn't know where the next allocated data region is?
> 	I'd specify this on these lines:
> 	i. If the DS has full knowledge of the allocation state for the whole file
> 	   the it SHOULD return the data_offset and data_length of the next allocated
> 	   region in the file.
> 	ii. Otherwise, if the next allocated offset is on the same strip unit as the hole, i.e.
> 	    it is served by the same DS, then the DS SHOULD return that data_offset, and set
> 	    data length to the number of bytes contiguously allocated following data_offset in
>             this stripe unit.
> 	iii. Otherwise, the DS SHOULD return the data_offset of the next stripe unit and set
>              data_length to zero.
> 
> Benny
> 
>

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 30 Sep 2010 15:16:27 -0400

> is how the client can use data_offset to improve performance.

Knowing that there may be nonzero values in the range
[data_offset,data_offset+data_length] isn't very useful--you're still
going to have to perform reads to get the data in that range.

The information that really *is* useful is the information that all data
in the range [read offset,data_offset] is zero.

So data_length doesn't give the client any new information it can use.

Simpler, I think, would be to either drop data_length entirely, or to
return a range describing the extent of the current *hole*, instead of
the extent of the next *non-hole*.

And just write the spec to say that the server return means that a read
of the returned range would have returned all zeroes.

--b.

---

From: Dean Hildebrand <[email-masked]>
Date: Thu, 30 Sep 2010 12:23:56 -0700

> of the returned range would have returned all zeroes.

I think the original motivation was for the client to adapt its requests 
using this field, so it wouldn't try to read into a hole.  So instead of 
reading 64K, which might go past data_offset+data_length, it simply 
reads the appropriate amount of data up until data_offset+data_length.  
Now, the server can always return a short read, but that does put the 
onus on the server to do fine-grained checking of zero and non-zero data 
for every read request, which may be quite onerous.  So I still think it 
has a purpose and should be kept in the draft.

btw, I like your comment regarding allocated and unallocated data.  I'll 
use that as motivation and refer to zero and non-zero data in the 
protocol description.

Dean

---

From: Dean Hildebrand <[email-masked]>
Date: Thu, 30 Sep 2010 13:14:55 -0700

>        file from data_offset until data_offset+data_length is allocated"
thanks.
 >Sorry, the strong semantics are desired for the range up until 
data_offset, not <data_offset, data_offset - data_length>

>This means that the area between 0x1000 and 0x4000 is a hole.

I agree with your comments and text.  I'm going to delete that line of text, as it is way too vague.

In fact, in your example. the area between 0 and 0x4000 are zero.

I'm going to put in a much more detailed description of these 2 fields, based on the excellent feedback we are getting.

> 	iii. Otherwise, the DS SHOULD return the data_offset of the next stripe unit and set
Thanks, this is pretty clear.  I think we can combine i and ii, as I 
don't think it matters really if the next non-zero region is on the same 
DS or another one, either way, it is the next non-zero region.

i. If the DS knows the offset and length of the next non-zero region, then it
    SHOULD return the data_offset and data_length of the next non-zero region in the file.
ii. If the DS only knows the offset of the next non-zero region, or is unaware of any
     information beyond the current stripe unit, the DS SHOULD return the data_offset
     of the next stripe unit and set data_length to zero.

I'll add this type of description into the non-pnfs case as well.
Thanks for you comments and text
Dean

---

From: "J. Bruce Fields" <[email-masked]>
Date: Thu, 30 Sep 2010 16:33:55 -0400

On Thu, Sep 30, 2010 at 12:23:56PM -0700, Dean Hildebrand wrote:
> I think the original motivation was for the client to adapt its
> requests using this field, so it wouldn't try to read into a hole.
> So instead of reading 64K, which might go past
> data_offset+data_length, it simply reads the appropriate amount of
> data up until data_offset+data_length.  Now, the server can always
> return a short read, but that does put the onus on the server to do
> fine-grained checking of zero and non-zero data for every read
> request, which may be quite onerous.

I don't understand.  Either the server finds out where the holes are, or
it doesn't--up to it.  If it knows where they are, then returning a
short read isn't any more or less hard than returning the correct
data_length would have been on the earlier read.

Or it could also just return the zeroes at the end of the read that
crosses the beginning of the next hole.  That doesn't strike me as a big
problem.  The client will still find out about a hole on its next read.

But, sure, I can believe that the client aligning its reads to with
respect to the possible boundary at data_offset+data_length could have
some benefit, and I didn't understand that before--thanks for pointing
it out.

There's still some question to me how beneficial that really would
be....

> So I still think it has a
> purpose and should be kept in the draft.
> 
> btw, I like your comment regarding allocated and unallocated data.
> I'll use that as motivation and refer to zero and non-zero data in
> the protocol description.

OK, thanks.

--b.