Subject: Re: READDIR cleanup

From: Peter Staubach <[email-masked]>
Date: Tue, 2 Nov 1999 15:57:07 -0800 (PST)

> 
> I would like to clear up some READDIR issues that have
> come up in WG email and during the bakeoff.
> 
> The READDIR arguments currently contain:
> 	cookie
> 	dircount
> 	maxcount
> 	attr_request
> 
> The response is:
> 	A linked list of:
> 		cookie
> 		name
> 		attrs
> 	eof
> 
> From NFSv3 we have lost the cookieverifier as part of the request
> and response.  So the server doesn't have a method to confirm
> the cookie.  Is that okay?  I would think not.
> 

From my viewpoint, the cookieverifier was a failed experiment in NFS
Version 3.  The semantics were never well understood nor supported by
many local file systems.  The Solaris NFS server always returns zeros
in the cookieverifier field.

> The dircount/maxcount is confusing or problematic.  Dircount is 
> supposed to be the size to read as represented in 'local' format.
> The maxcount is supposed to be the encoded XDR stream.
> Is this useful?  I would think the client would like to find
> the size of the directory with GETATTR and then do a read
> to either read the entire directory's contents or just a portion
> of the directory (if it is reather large).  Is dircount sufficient
> to represent the amount of the directory to read?  I think so
> if dircount is defined as the size of the directory at the
> server's terms.
> 

What is confusing about these two fields?

Dircount is intended to be a hint to the server as to the number of
bytes of directory information which is desired.  It is of limited
usefulness because different clients have different ideas on what
directory entries actually look like, but it does have some
usefulness.  However, between some client/server combinations, it is
extremely handy.

The maxcount field was added in order to put an upper bound on the
amount of XDR encoded information which can be returned.  For example,
the response to a READDIR request for 1048 bytes of directory
information may contain just a few entries or it may a lot of entries
if the individual names are small.

By the way, doing a GETATTR on a directory and then attempting to read
that many bytes of directory information is completely bogus and should
be avoided at all costs.  Even on like UNIX systems, dirents don't look
like directs.

> The intent behind maxcount is that it is a way for the client
> to place an upper limit on the XDR data stream that is returned.
> The maxcount semantics places encoding complexities on the server
> that may cause bugs in the server implementations.  If the server
> makes a mistake and encodes too much, what will the client do
> with the return value?  I would suggest removing maxcount and
> keeping the semantic that the server can return less data on
> the READDIR than was requested.
> 
> Another approach to readdir would be to have the dircount
> field be the number of entries to return and not the size.
> The problem with this is that the client doesn't have a way
> to determine how many entries are in the directory to enable
> reading the entire directory in one request.  I don't think
> this is a viable option.  There are other problems with APIs
> at the server that would easily enable this type of semantic.
> 

So, if you don't have a dircount like element, how much data should
the server attempt to read from the directory in order to satisfy the
READDIR request?  A lot?  A little?  Picking the wrong number can be
dreadful, performance wise.  At least with some sort of hint, the
server has a starting place.

> The last issue relates to the encoding of the result.
> It was suggested that a counted array may be a reasonable
> encoding instead of the linked list?  I like the idea of
> the counted array but will this cause extra overhead at the
> server?  It certainly would with the current maxcount
> semantic
> 	

Is there a problem with the linked list encoding which people have
not already solved years ago with their NFS Version 2 and Version 3
implementations?  Are we solving a non-problem here?

	Thanx...

		ps

---

From: Spencer Shepler <[email-masked]>
Date: Tue, 02 Nov 1999 18:41:02 -0600

> in the cookieverifier field.

The server will rely on its internal tracking of cookies it provides
to determine if it should return NFS4ERR_BAD_COOKIE for a particular
use of a cookie value by a client.  The client will then need to 
deal with the BAD_COOKIE error.  I will need to add a description of what
the client is supposed to do with a BAD_COOKIE error.

> usefulness.  However, between some client/server combinations, it is

Then I will need to add the fact that it is a hint because of 
the differences.

> implementations?  Are we solving a non-problem here?

Issue was raised outside of the WG alias; trying to bring it to closure.

Spencer

---

From: Mike Eisler <[email-masked]>
Date: Tue, 2 Nov 1999 16:54:21 -0800 (PST)

> in the cookieverifier field.

If we are supporting file set migration in the NFS V4 protocol, don't we need
the cookieverifier to allow the secondary server to indicate that the cookies
used before won't work? Simply declaring that after a migration, the
cookies can no longer be used is inflexible ... some file set transfer
can preserve cookies, such as moving removable media from one server
to another.

> usefulness.  However, between some client/server combinations, it is

Given that NFS V4 is intended to be more hetereogeneous than
NFS V3, that doesn't seem like sufficient grounds for retaining dircount.

What would be the consequence between "some client/server combinations" if
the only thing transmitted is a maxcount?

> if the individual names are small.

The confusion is whether either a hint, whether it is fatal to go beyond the
hint, and if maxcount is a maximum bound on XDR encoded data, then why don't
implementations use xdr_sizeof()?

My preference would have been for nfs v4 to uses a directory entry count in
the arguments, and a variable length array in the results. However, as I said
at the bakeoff, it's probably too late for that degree of change. I suggest
declaring dircount and maxcount as hints,  leave it to server to whether the
hints apply to unencoded or encoded data.

	-mre