Subject: RE: tar for NFS V4

From: Eric Kadison <[email-masked]>
Date: Fri, 31 May 2002 10:20:59 -0700

I agree that a joint effort between the two communities is a very good
long-term idea.  My immediate interest was short term.  tar is currently
used as both a backup program and a standard archive format for
distribution.  It seems to me that customers would want a new version of tar
that:

1.  is aware of the new attributes available under NFS V4.  e.g. tar can
currently set the last modified time of a file during extraction.  Wouldn't
it be desirable to also be able to set the last backup time?  Aren't there
other new attributes, e.g. archive, hidden, in the same class.  

2.  is backward compatible with pre-NFS V4 file systems and existing tar
balls.

3.  is public domain, like GNU tar.

Furthermore, GNU tar has a special ability to archive sparse files (I don't
know how it knows sparseness).  Does that method continue to work under NFS
V4?

Eric

> -----Original Message-----
> From: Robert Thurlow [mailto:[email-masked]]
> Sent: Friday, May 31, 2002 9:42 AM
> To: Skardal, Harald
> Cc: 'Gordon Waidhofer'; Harris, Guy; Eric Kadison;
> [email-masked]; [email-masked]
> Subject: RE: tar for NFS V4
> 
> 
> > Just as the the NFSv4 community is going outside its
> > traditional scope and getting interested in file system
> > migration, I think the NDMP cmmunity have to take a step outside
> > our traditional turf and consider seriously the task 
> > of backing up date with richer metadata, such as NFSv4, 
> > NTFS etc.
> 
> Correct me if I'm wrong here Harald, but doesn't NDMP support
> these things if only because the server owing the files gets
> to decide the backup data stream stored and retrived, and can
> put richer metadata into that stream?
> 
> That said, I'd like to see a definition of a canonical data
> format that many servers can understand, so that they can
> exchange data more readily.  I like the AFS model - on any
> architecture AFS supports, you can get a bytestream which
> includes all data and metadata AFS can store, and you can
> then put it out on tape as a backup, restore it, or use it
> to do replication and migration.  With NDMP+security fixes,
> a good canonical (and extensible) data format could solve
> the replication and migration problem for NFSv4 and other
> areas.  As a later goal, this could also be replacement for
> tar/cpio/pax and friends.
> 
> > I could see this being a "joint venture" between NDMP and NFS,
> > and possibly others. (WebDAV?). It seems as if there are 
> > two deliverables, a data format specification, and an open
> > source implementation for Linux.
> > 
> > First question: What is the general interest level in the NFSv4
> > group and in the NDMP collaborative in solving this problem 
> > together?
> 
> I'd be much in favor of it.  We have some problems logistically
> until the Network Backup and Restore (NBR) workgroup is approved
> by the IETF, but after that, I think much is possible.  Clearly,
> we need this not to be too much of a distraction to the NBR WG.
> 
> Is anyone interested in a bar BOF in Yokohama for this stuff?
> 
> Rob T
>

---

From: Robert Thurlow <[email-masked]>
Date: Fri, 31 May 2002 11:48:07 -0600 (MDT)

> distribution.

I understand.  It would be good if someone simply tried to
modify GNU tar to support some of the extra stuff.  The easy
part would be to extend the archive format; harder is the
code to suck all of this out of any particular local filesystem :-)

Rob T

---

From: "Gordon Waidhofer" <[email-masked]>
Date: Fri, 31 May 2002 11:00:20 -0700

Harder still is to cook up an archive extension
method that doesn't make unextended tar's cough
up fur balls. GNU tar has a few tricks (long file
names, sparse files, etc). But you still get fur
balls when you extract with an old tar. And it
looks like it won't be long before today's GNU
is the unextended, fur ball coughing tar. And then
there are present day extensions like Solaris 9.
Such are future legacy at least. I suppose a
certain amount of divergence before convergence
is to be expected.

This idea of migration stream format and backup
data format having commonality is most intriguing.
Nice catch Brent and Rob.

	-gww

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Robert Thurlow
> Sent: Friday, May 31, 2002 10:48 AM
> To: Eric Kadison
> Cc: [email-masked]'

---

From: Guy Harris <[email-masked]>
Date: Fri, 31 May 2002 12:05:02 -0700

On Fri, May 31, 2002 at 10:20:59AM -0700, Eric Kadison wrote:
> it be desirable to also be able to set the last backup time?

Yes, it would.

What's the OS API to get and set the last backup time on those UNIXes
that support NFS v4? :-)

(I.e., my original comment still applies.  The mere fact that NFSv4
supports a particular feature does *not* mean that all OSes supporting
NFSv4 magically support that feature.)

> know how it knows sparseness).

GNU tar 1.11.8, at least, infers the sparseness of the file by checking
the file size against the number of blocks allocated for it.