Subject: Byzantine Routers - WRITE operations

From: Neil Brown <[email-masked]>
Date: Thu, 18 Nov 1999 11:32:04 +1100 (EST)

As I mentioned, I have been pondering the effects of Byzantine Routers
(BR) and atmost-once semantics and I think there are some interesting
issues with delayed WRITE operations.

The WRITE operation has a stateid so that, for example, a write to a
locked region will not inadvertently succeed after the region has been
unlocked.

However there are two scenarios that I have thought of that aren't so
well protected.

One involves write delegations.  If I have a write delegation, then I
will (I presume) use the write-delegation stateid for all my write
requests.
If I have a write delegation for a long time I would probably write
back cached data fairly lazily, but I wouldn't hold it forever.  
If my delegated file received random writes, it is possible that there
would be an update for a block that has already been written once.
This would mean that the one block of data would be written to the
server twice with the same stateid.  This could happen even if the
application had done all the locking that it should have done.  And
this is exactly the sort of situation that the BR threatens.

I think this is solvable.  Does it need to be solved?

The other scenario is more difficult.  
It has been acknowledged that some applications use locks as
semaphores for access to other files or byte ranges that are not
locked.
If this approach is being used for concurreny control, then the
stateid in the WRITE request provides no guarantee that the relevant
lock is still in place.  Solving this would require single threading
all IO with all locks, which wouldn't be popular.  I think the only
solution is to assert that this problem is unsolvable, and to
encourage developers to use locks when they want locks.

NeilBrown