Subject: RE: fsync() fails under NFS, right?

From: "Noveck, Dave" <[email-masked]>
Date: Mon, 13 Sep 1999 11:02:21 -0700

> 
> > I don't see anything in the spec that says that a
> > WRITE has to be atomic in the sense that a READ cannot
> > see an in-progress partially-completed WRITE.  You can
> > use file locking if you need that kind of guarantee.
> 
> True. Though the original poster did not raise this
> question, so I'm taking it for granted that they have
> exclusion under control. Again, this is not peculiar to NFS.

Not necessarily.  Many local file systems are implemented
so that individual reads and write calls are atomic as
far as other accessors of the file are concerned, even
without locking.  Nfs has never had that guarantee but
I think that people in the past considered that fact part
and parcel of the cache-consistency issue which was going 
to be resolved "some day".  I just want us to be clear 
that you don't have the exclusion guarantee (without 
locking) even at the level of individual requests (i.e.
that this lack of exclusion semantics is not an artifact
of the nfs laissez-faire caching model).

> 
> > The big problem is atomicity in the face of failures.
> > Locking doesn't help you here.  As far the protocol
> > goes, there is no guarantee that an unacknowledged
> > WRITE is either completely-done or not-done-at-all
> > after a server failure.  Many servers have the
> > property that this is true for some range of write
> > sizes and alignments and some servers can provide it
> > for all writes but it is not a protocol reqruirement
> > as far as I know.
> 
> This issue exists whether NFS, UFS, or straight disk.
> It's the damaged sector problem. I'll bet that the
> design the original poster is working with already
> takes this into account. If a "root" (superblock)
> is flawed, the previous root is used.

The issue may be different here.  The original poster
said "sector-sized".  From that I supposed that he
was assuming that a partial write of a sector would
show up as bad in the sense that the disk would give
you an IO error (due to CRC).  That may happen on the
nfs server but you can't assume that, since you don't
know how the server arranges data within sectors.
So in that case, you need to be able to detect a
damaged block yourself using something like an
application-level checksum.

> 
> > Making that a protocol requirement would be too onerous
> > in general.  I think it would be good idea to give the
> > server a way to inform the client of what atomicity
> > guarantees he provides.  A per-fsid attribute would
> > be nice, if we can figure out what would go in it.
> 
> Just an extended attribute(s) on the actual file would
> be fine. Some file services may have special provisions
> in their file model for such a requirement. As such,
> the attribute would not be fsid wide, but rather
> file specific.

I don't understand how a server would provide different
atomicity guarantees for different files in the same
file system.  My model is that the server is going to 
simply advertise the normal capabilities that he has.
He wouldn't accept any special (per-file or otherwise)
atomicity requests.  That starts to get difficult.

> Again, I don't think the poster is asking for the protocol
> to support transactions, ala Seltzer et al. Rather,
> just that the normal POSIX/UNIX semantics for fsync()
> be dependable over NFS. As such, all other issues
> of ordering, fault tolerance, and exclusion are likely
> addressed by their design. Just an educated guess, though.
 
I just want to be clear what nfs provides and what it 
doesn't.  In some cases, there are guarantees that typical
unix file systems provide that the nfs protocol doesn't 
provide, even though most servers may provide that same 
guarantee.

---

From: Alexy Khrabrov <[email-masked]>
Date: Tue, 14 Sep 1999 13:33:54 -0400 (EDT)

Greetings --

I was away from my mail, but want to thank all who
responded.  For us, the issue is very crucial, and
it is interesting that it doesn't look as such to
many NFS folks.  The "educated guesses" about our
model are remarkably correct.

What surpises me is that I get an impression
that most NFS folks assume fsync() is supposed to
work under NFS.  However, a dilettante I am, only
having read Tanenbaum book, I always assumed NFS
is stateless -- it doesn't maintain cursors for
each user, then how can it do fsync()??  Flush all
I/O on the file?

Thus, I'd like to hear what do NFS people say
think -- fsync() must work under NFS, if it
doesn't, it's a particular client bug, a correct
server allows fsync() to work, and there's no
theoretical problems preventing NFS model from
having totally operable fsync().  Is that right?

-- 
Cheers,
Alexy Khrabrov -- www.suffix.com -- Segmentation f%^(&