Subject: Re: [nfsv4] pnfs: read/write through MDS hint (Issue 55)

From: Dean Hildebrand <[email-masked]>
Date: Tue, 01 Aug 2006 16:30:20 -0400

>  For either
> threshold type, a value of 0 indicates no read or write should be
> issued to the metadata server, while a value of ~0 indicates all reads
> or writes should be issued to the metadata server.
Do you mean all 1s, or do I not understand a common term (~0)?
>
> Due to dynamic system changes, the client
> should not assume that the attribute will remain constant for any
> specific time period, thus it should be periodically refreshed.
Should we specify some minimum amount of time that it is valid (such as 
while a file is open) ?
Dean
>
>
> Structures:
> -----------
>
> struct pnfs_mdsthreshold_item {
>   pnfs_layouttype4  layout_type;
>   bitmap4           hintset;
>   opaque            hintlist<>;
> };
>
> struct pnfs_mdsthreshold {
>   pnfs_mdsthreshold_item hints<>;
> };
>
> The pnfs_mdsthreshold structure holds an array of hints that can help
> the client determine when it should issue I/O directly through the
> metadata server vs. the data servers.  Each hint structure is specific
> to a layout type.  Since an implementation may support multiple layout
> types, an array of these structure is needed.  The hint structure
> consists of the layout type, a bitmap describing the set of hints
> supported by the server, they may differ based on the layout type, and
> a list of hints, whose structure is determined by the hintset bitmap.
>
>
>
> Bitmap descriptions for the hintlist field:
>
> Name                   #  Data Type Access  Description
> ----                   -  --------- ------  -----------
> read_size_threshold    1  length4   READ    The file size below which it
>                                             is recommended to read data
>                                             through the MDS.
>
> write_size_threshold   2  length4   READ    The file size below which it
>                                             is recommended to write data
>                                             through the MDS
>
> read_iosize_threshold  3  length4   READ    For read I/O sizes below
>                                             this threshold it is
>                                             recommended to read data
>                                             through the MDS
>
> write_iosize_threshold 4  length4  READ     For write I/O sizes below
>                                             this threshold it is
>                                             recommended to write data
>                                             through the MDS
>
> Marc Eshel wrote:
>> I would be happy with the simple struct if we added a separate 
>> attribute for read and write.
>>
>>  struct pnfs_mdsthreshold_item {
>>     pnfs_layouttype4  layout_type;
>>     length4           read_size_threshold;
>>     length4           write_size_threshold;
>>  };
>>
>> Marc.
>>
>> Garth Goodson <[email-masked]> wrote on 06/14/2006 03:35:15 
>> PM:
>>
>>> There has been some discussion about a hint that can be used to help 
>>> a client decide whether or not to read/write through the MDS vs the 
>>> DS. I've described two types that could be used for these hints.  
>>> The first is fairly basic, while the second is a little more 
>>> complex.  The motivation for the second is to provide a way to 
>>> extend this attribute in the future.
>>>
>>> 1. Simple structure:
>>> --------------------
>>>
>>> struct pnfs_mdsthreshold_item {
>>>    pnfs_layouttype4  layout_type;
>>>    length4           size_threshold;
>>> };
>>>
>>> struct pnfs_mdsthreshold {
>>>    pnfs_mdsthreshold_item hints<>;
>>> };
>>>
>>>
>>> 2. More complex, more extensible:
>>> ---------------------------------
>>>
>>> struct pnfs_mdsthreshold_item {
>>>    pnfs_layouttype4  layout_type;
>>>    bitmap4           hintset;              /* Similar to
>>>    opaque            hintlist<>;              fattr4 */
>>> };
>>>
>>> struct pnfs_mdsthreshold {
>>>    pnfs_mdsthreshold_item hints<>;
>>> };
>>>
>>> Bitmap descriptions:
>>>
>>> Name            #  Data Type   Access   Description
>>> ----            -  ---------   ------   -----------
>>> size_threshold  1  length4     READ     The file size below which it is
>>>                                          recommended to read/write data
>>>                                          through the MDS
>>>
>>> Description
>>> -----------
>>>
>>> This attribute acts as a hint to the client to help it determine 
>>> when it 
>>
>>> is more efficient to read/write a file to the metadata server vs. 
>>> the dataserver.  Regardless of the structural representation, the 
>>> meaning of 
>>
>>> the size_threshold is the same.  Files with a size shorter than 
>>> 'size_threshold' should be read/written to the MDS; all other files 
>>> should be read/written to the data servers.
>>>
>>> As Benny pointed out in an earlier email:
>>>
>>> If the size_threshold is set to 0 then no files (shorter than or 
>>> equal to 0 bytes) will be read from/written to the MDS, if it's set 
>>> to ~0 then 
>>
>>> all files (shorter than or equal to ~0 bytes) will be read 
>>> from/written to the MDS.
>>>
>>> An array of hints based on layout type is used, as a server may 
>>> support multiple layout types and may want to be able to return 
>>> different attribute values depending on the layout type.  In the 
>>> common case, this 
>>
>>> will be a short list.
>>>
>>> A file or FSID attribute?
>>> -------------------------
>>> There have been arguments made both ways.  An FSID attribute allows 
>>> a client to query it, before the file is open.  However, one could 
>>> always query the attribute prior to performing an I/O.
>>>
>>> An alternate solution is to make the attribute available both as a 
>>> FSID and a file attribute and allow the server to decide if files 
>>> have different attribute values than the file system.  Thus, a query 
>>> using a file's file handle may return the generic file system 
>>> attribute or a specific attribute for only that file.  If the client 
>>> wants a generic file system attribute it should not call the GETATTR 
>>> with a file 
>> filehandle.
>>> Comments?
>>>
>>> Thanks,
>>> -Garth
>>>
>>> _______________________________________________
>>> nfsv4 mailing list
>>> [email-masked]
>>> https://www1.ietf.org/mailman/listinfo/nfsv4
>
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/nfsv4

-- 
Dean Hildebrand
Ph.D. Candidate
University of Michigan

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Garth Goodson <[email-masked]>
Date: Tue, 01 Aug 2006 16:09:50 -0700

> Do you mean all 1s, or do I not understand a common term (~0)?

Yes, all ones.

> Should we specify some minimum amount of time that it is valid (such as 

This is tricky.  I don't think we can impose a minimum on an 
implementation.  Some apps may keep files open for a very long period of 
time, and it would be unreasonable to restrict the server from 
optimizing itself to adhere to this restriction.  We talked a bit about 
this in Montreal and there is no good solution at this time.  Some 
prefer the client to poll, while others prefer some type of callback.

-Garth