Subject: Re: Replication and Migration - summary so far

From: Brent <[email-masked]>
Date: Mon, 01 Oct 2001 11:09:57 -0700

Robert Thurlow wrote:
> We want to create a new protocol which servers can use to transfer
> and maintain replicas of NFSv4 filesystems, or to migrate NFSv4
> filesystems from one server to another.  The protocol should likely
> have most of the following attributes: heterogeneous, correct, fast,
> secure, and bandwidth efficient.

Correct - except for "NFSv4".  This protocol should be about
filesystems, period - not NFSv4.  If it works for NFSv4 too,
that's great.  A filesystem migration/replication protocol
should be useful for all filesystems, whether served by
NFSv4 or not.

> Delegation and lock state
> Migration of a read-write filesystem needs either to transfer
> state for current delegations and locks or ensure that such
> state does not exist when the transfer starts.  This is seen
> as difficult to do, so some want to ignore this for now and
> focus on what we need to make read-only replication work well.
> Compromises might be to leave a place holder in the protocol
> for this unique metadata and define how it is to be handled
> later, or to make this a SHOULD in the protocol.
> 
> Question to the WG: how important does it seem to you to
> solve this problem?  Does either compromise work for you?

A migration/replication protocol would be very useful even
without file open/lock state transfer.  NFSv4 is already
designed to provide graceful recovery from state loss.
I'd leave state transfer discussions for "Phase II."

> NFSv4-specific vs. generic
> Brian's original work had some language that this would not
> be a general purpose replication protocol for use with older
> versions of NFS and with CIFS, but others want exactly that
> sort of capability.  It does seem that without knowing some
> intimate things about NFSv4 metadata, the protocol cannot
> truly transfer filesystems with the degree of correctness
> desired.
> 
> Spencer Shepler suggested what I think is an excellent way
> to handle this: that rather than putting core knowledge of
> NFSv4 into the replication/migration protocol, we define a
> series of protocol-specific "attachments" which contain the
> details of how metadata looks, and make the first attachment
> for NFSv4, able to use NFSv4 definitions.
> 
> Question to the WG: does this kind of layering make sense?

This is a good approach, though I'd make the attachments
specific to filesystems - not protocols.  The canonical
approach (as we've attempted for NFSv4) won't provide
good enough translation of metadata - too much information
is lost.  I recommend direct filesystem-filesystem metadata
translation. Good initial metadata attachments might be generic
POSIX nd Windows filesystems.  

This needn't develop into an "n x n translation" problem.
Filesystems fit quite well into property inheritance 
trees, e.g. UNIX/Linux rooted at POSIX, so I think the
metadata translation problem is manageable.  You'll
certainly get perfect metadata translation between
like filesystems.

> Capability negotiation
> Brian's original talks had a negotiation phase between the
> peers of a filesystem transfer so that the sender can know
> any impairments before starting an initial transfer and
> warn an admin or bail out.   Some weren't sure this was
> necessary.  But Dave Noveck wrote a good argument for this,
> stating that portable programs often try to learn their
> environments before doing serious work, and that losing
> an extended attribute after a migration event would be a
> very bad thing.
> 
> Question to the WG: do we need capability negotiation?

Sure, I think it's reasonable to be able to determine
the quality of the migration/replication before it's
attempted.

> We also have some things that seem to be accepted based on
> the amount of attention paid to them so far.  I think the
> following are not in much dispute; please argue with me if
> you disagree:
> 
> - it is OK for the replication/migration protocol to not be
>   based on RPC
> - it is OK for the replication/migration protocol to be able
>   to rely on transport-level security (e.g. IPSEC)
> - it is likely interesting to use the "rsync" algorithm to
>   transfer replicas in some cases, though there will be
>   cases where we have, and should use, better information
>   from the filesystem
> - resource discovery (how the client finds the filesystems
>   to use) is impacted by replication and migration, but is
>   not part of this solution space
> - multiple writable copies and disconnected operation are
>   not in scope

I agree with all of these.

	Thanks
		Brent