Subject: RE: Caching issue (SHORT, really, I swear)

From: "Noveck, Dave" <[email-masked]>
Date: Wed, 11 Aug 1999 13:49:01 -0700

Nice to have you join the discussion.  Mike and I were getting
*so* lonely. :-)

> Based on my understanding of the discussion so far, I have two
> questions.
> 
> 1. If the client has a write delegation, does it still have to flush
> to the server on CLOSE?

Yes, sometimes.  That sometimes is the nub of the discussion.  In
my proposal, files whose size is below a limit specified by the
server will avoid having to be flushed on close.  Mike thinks that
for the environments that he is concerned with that sometimes 
will be always or at least far too often.  My argument is that for
many environments sometimes will be much less than always and that
this is a win.  I say it's half-a-loaf better than nothing and
Mike says its a measly worthless slice and not worth bothering
with.

I hope that's a fair summary. 

> 
> 2. What would happen if Dave's proposal were modified so that instead
> of the server expressing a limit in terms of file size, the server
> expresses the limit in terms of how much unwritten (dirty) data the
> client can accumulate before writing back to the server?

My answer is it that it would cease to work.  The problem is that
the mapping from amount of dirty data to amount of disk depends 
(in a large way) on where specifically, the data is to be written.
If I am allowed to write 1000 bytes, if I were to write one thousand
very dispersed one-byte areas, the amount of space required might be
enormous and it would depend on lots of things, depending on the
type of server.  Things like indirect block structures and how holes are
handled would all come into play.

It is possible that we can come up with some way to deal with these
issues that is not horribly complex.  I'm hoping we can.  Right now
my focus is on getting the caching framework into the proposal.
I think that's a big win so people can incorporate delegation
into their prototype implementations and we still have the chance
to make some limited localized changes to try to do better in this 
particular area.  All volunteers for that effort gratefully accepted.

> 
> (Apologies if I'm going back over ground that was already covered in
> earlier discussions.  Pointers to archived mail that I should (re)read
> are cheerfully accepted.)
 
We have dealt with so many issues in each of the long emails that 
we have exchanged that responding directly is easier than trying to
find all of the relevant background.

---

From: James Yarbrough <[email-masked]>
Date: Wed, 11 Aug 1999 15:29:54 -0700

> handled would all come into play.

I've come to agree that the file size limit for write caching is too
restrictive and that a flush on close requirement would be better.
I also don't see a way for the server to specify an a priori limit
other than by file size.

Here is my understanding of the problem in the context of the current
proposal:

	1) when a write open delegation is granted to a client, the
	   client is not required to a close or write back data (up to
	   a specified file size);
	2) space availability (or lack thereof) for dirty data is
	   guaranteed to be determined by close time at the latest;
	3) since closes are not being sent to the server, the client
	   must be able to determine on its own what to do with dirty
	   data.

From these three things comes the file size limitation in write
delegations.

Is my statement of the problem correct?

-- 
James Yarbrough

[email-masked]

Blues, an important part of a balanced psyche.

---

From: Mike Eisler <[email-masked]>
Date: Wed, 11 Aug 1999 15:50:14 -0700 (PDT)

> enormous and it would depend on lots of things, depending on the

So why can't the server report the number of bytes the client is
allowed to cache and also what the block size is?

> handled would all come into play.

Issues of indirect blocks apply whether one is using a limit in terms of file
size or block count. As for holes maybe it means that we don't support
write delegation on file systems that don't efficiently deal with holes, or
the limit is a discriminated union that's either byte count or file length.

	-mre

---

From: Mike Kupfer <[email-masked]>
Date: Wed, 11 Aug 1999 16:29:08 -0700

My high-order concern was that a client and server on a LAN not take a
performance hit when they go from v3 to v4.  I think your proposal
satisfies that concern, even if the client has to write back on CLOSE.
(Sorry if this was obvious to everyone else and I just figured it out.)

As for the space reservation scheme, I see two unresolved questions.

1. what's the right metric: file size, number of dirty bytes, or
  something else.  

2. (As far as I know) we don't have a good understanding of how the
  server calculates and reports reservations in the face of multiple
  open files and multiple clients.  I think the Spritely NFS folks
  dealt with this some, though I don't remember the details or which
  paper discussed it.

So as far as any straw polls, I'm inclined to leave space reservation
out for now.  Then if we have time and ideas, we can try to tackle the
two questions above.

mike

---

From: Ted Anderson <[email-masked]>
Date: Wed, 11 Aug 1999 20:23:53 -0400 (EDT)

I am afraid that the write back on close is not a panacea because on
Windows clients errors detected at this point cannot be reported to the
user application.  Quota/space errors are expected to be reported in the
context of write calls.  Perhaps someone familiar with Windows NFS
integration can comment on this in more detail.  In DFS, which does
write back on close, we have found that various (dubious) tricks are
necessary to avoid this problem.  To do a better job it be nice to have
some kind of reservation system so that the client can be reasonably
sure that the server can accept all the dirty data it has.  This leads
to a scheme like a) or c).  Because of the Windows problem I don't think
b) make a great fallback.  I think that the many advantages of write
delegation make d) a bad choice.

Ted Anderson