Subject: [nfsv4] FedFS Meeting Minutes, 3/11/2010

From: James Lentini <[email-masked]>
Date: Thu, 11 Mar 2010 14:43:54 -0500 (EST)

FedFS Meeting Minutes, 3/11/2010
--------------------------------

Attendees
---------

Craig Everhart (NetApp) 
Sorin Faibish (EMC)
Chuck Lever (Oracle)
James Lentini (NetApp)
Trond Myklebust (NetApp)
Renu Tewari (IBM)
Chris Stacey (EMC)

Minutes
-------

+ IETF Note Well Agreement

  This is a reminder that our discussions are governed by the 
  IETF Note Well Agreement. See:

    http://www.ietf.org/NOTEWELL.html

  We will start each week's meeting with this announcement.

+ IETF'77

  James has requested presentation slot.

  The presentation will detail our status and plans for each of the 
  drafts.

  James will draft slides and circulate with the group next week.

+ Admin Protocol Review

  At the end of last week's meeting, we discussed ways to compress the 
  arguments to FEDFS_CREATE_JUNCTION.

  James sketched out one possible approach below. In this approach, the 
  Admin client assists the fileserver by identifying the junction create 
  parameters (NSDB name, LDAP NSDB Container Entry, and security parameters) 
  with an opaque handle. The goal is to simplify the fileserver at the 
  expense of a more sophisticated protocol.

  Another approach (not shown) would be for the fileserver to 
  internally compress/deduplicate the parameters to junction 
  create. 

  We first reviewed the concerns raised last week. The concern (as we 
  understood it) was that the FSN data (UUID, NSDB's DNS name, and 
  LDAP search base) would be too large to store in a junction.

  We reviewed these two approaches.

  Chuck suggested that the fileserver could store FSN values 
  separate from the junction. The junction would contain a pointer 
  to the appropriate FSN. The implementation would be transparent to clients 
  and not exposed in the Admin protocol. file or database. 

  An example implementation would be for junction information to be 
  stored in an extended attribute. The junction information could indicate 
  the location of the FSN in a separate file or database.

  James noted that this was the second approach outlined above.

  We reviewed the first approach (shown below). James noted that he 
  was concerned with extra complexity necessary on the client and 
  asked for suggestions or other approaches.

  We agreed that the best way forward on this topic would be to get 
  more details on implementations that have limitations on the FSN 
  size.

  We then discussed how the administrator would interact with the 
  system.

  We discussed if using a URL in place of a (DNS name, NSDB Container 
  Entry).

  We discussed what a UI tool could do. The administrator won't want to 
  write down UUIDs or NCE strings. A GUI could allow the administrator to 
  select and NSDB and FSN, and then the tool would be able to extract any
  low level arguments like UUID, NCE, etc.

-------

enum FedFsStatus {
 FEDFS_OK                 = 0,
 FEDFS_ERR_ACCESS         = 1,
 FEDFS_ERR_BADCHAR        = 2,
 FEDFS_ERR_BADXDR         = 3,
 FEDFS_ERR_EXIST          = 4,
 FEDFS_ERR_INVAL          = 5,
 FEDFS_ERR_IO             = 6,
 FEDFS_ERR_NOSPC          = 7,
 FEDFS_ERR_NOTDIR         = 8,
 FEDFS_ERR_NOTEMPTY       = 9,
 FEDFS_ERR_NOTJUNCT       = 10,
 FEDFS_ERR_NOTLOCAL       = 11,
 FEDFS_ERR_PERM           = 12,
 FEDFS_ERR_ROFS           = 13,
 FEDFS_ERR_SVRFAULT       = 14,
 FEDFS_ERR_NSDB_ROUTE     = 15,
 FEDFS_ERR_NSDB_DOWN      = 16,
 FEDFS_ERR_NSDB_CONN      = 17,
 FEDFS_ERR_NSDB_AUTH      = 18,
 FEDFS_ERR_NSDB_LDAP      = 19,
 FEDFS_ERR_NSDB_NOFSN     = 20,
 FEDFS_ERR_NSDB_NOFSL     = 21,
 FEDFS_ERR_NSDB_RESPONSE  = 22,
 FEDFS_ERR_NSDB_FAULT     = 23,
 FEDFS_ERR_NSDB_PARAMS    = 24
};

typedef opaque FedFsUuid<16>;

typedef opaque FedFsNsdbName<>;
typedef opaque FedFsNsdbContainerEntry<>;
typedef opaque FedFsNsdbHandle<8>;

typedef opaque FedFsPathComponent<>;
typedef FedFsPathComponent FedFsPathName<>;

struct FedFsNsdb {
        FedFsNsdbName            name;
        FedFsNsdbContainerEntry  nce;
};

struct FedFsFsn {
        FedFsUuid         fsnUuid;
        FedFsNsdbHandle   handle;
};

struct FedFsCreateJunctionArgs {
        FedFsPathName     path;
        FedFsFsn          fsn;
};

enum FedFsResolveType {
 FEDFS_RESOLVE_NONE  = 0,
 FEDFS_RESOLVE_CACHE = 1,
 FEDFS_RESOLVE_NSDB  = 2
};

struct FedFsLookupFsnArgs {
        FedFsPathName          path;
        FedFsResolveType       resolve;
};

union FedFsResolveRes switch (FedFsResolveType resolve) {
 case FEDFS_RESOLVE_NONE:
        void;
 default: /* CACHE or NSDB */
        FedFsUuid              fslUuid<>;
};

struct FedFsLookupFsnResOk {
        FedFsFsn               fsn;
        FedFsResolveRes        resolveResult;
};

union FedFsLookupFsnRes switch (FedFsStatus status) {
 case FEDFS_OK:
        FedFsLookupFsnResOk    resok;
 case FEDFS_ERR_NSDB_LDAP:
        unsigned int           ldapResultCode<1>;
 default:
        void;
};

enum FedFsConnectionSec {
 FEDFS_SEC_NONE = 0,
 FEDFS_SEC_TLS = 1 /* StartTLS mechanism; RFC4513, Section 3 */
 /* other mechanisms TBD */
};

struct FedFsNsdbParams {
        FedFsConnectionSec       secType;
        /* type specific data follows */
        opaque                   secData<>;
};

struct FedFsSetNsdbParamsArgs {
        FedFsNsdb                nsdb;
        FedFsNsdbParams          params;
};

union FedFsSetNsdbParamsRes switch (FedFsStatus status) {
 case FEDFS_OK:
        FedFsNsdbHandle          handle;
 default:
        void;
};

union FedFsGetNsdbParamsArgs switch (bool byName) {
 case TRUE:
        FedFsNsdb                nsdb;
 case FALSE:
        FedFsNsdbHandle          handle;
};

struct FedFsGetNsdbParamsResOk {
        FedFsNsdb                nsdb;
        FedFsNsdbHandle          handle;
        FedFsNsdbParams          params;
};

union FedFsGetNsdbParamsRes switch (FedFsStatus status) {
 case FEDFS_OK:
        FedFsGetNsdbParamsResOk  resok;
 default:
        void;
};

struct FedFsGetLimitedNsdbParamsResOk {
        FedFsNsdb                nsdb;
        FedFsNsdbHandle          handle;
        FedFsConnectionSec       secType;
};

union FedFsGetLimitedNsdbParamsRes switch (FedFsStatus status) {
 case FEDFS_OK:
        FedFsGetLimitedNsdbParamsResOk  resok;
 default:
        void;
};

program FEDFS_PROG {
 version FEDFS_VERSION {
     void FEDFS_NULL(void) = 0;
     FedFsStatus FEDFS_CREATE_JUNCTION(
                  FedFsCreateJunctionArgs) = 1;
     FedFsStatus FEDFS_DELETE_JUNCTION(
                  FedFsPathName) = 2;
     FedFsLookupFsnRes FEDFS_LOOKUP_FSN(
                  FedFsLookupFsnArgs) = 3;
     FedFsSetNsdbParamsRes FEDFS_SET_NSDB_PARAMS(
                  FedFsSetNsdbParamsArgs) = 4;
     FedFsGetNsdbParamsRes FEDFS_GET_NSDB_PARAMS(
                  FedFsGetNsdbParamsArgs) = 5;
     FedFsGetLimitedNsdbParamsRes FEDFS_GET_LIMITED_NSDB_PARAMS(
                  FedFsGetNsdbParamsArgs) = 6;
 } = 1;
} = 100418;

---

From: "Everhart, Craig" <[email-masked]>
Date: Tue, 16 Mar 2010 12:11:56 -0400

Based on comments received since October, I'm proposing the following
changes to the document
draft-ietf-nfsv4-federated-fs-dns-srv-namespace-02.txt.  I keep hitting
the draft upload embargo window, but I want to review the comments at a
high level before uploading.

(1) Replace "domain-root" with "domainroot" in directory names, to match
the SRV Service label component.  So where we had 
	/.domain-root-{name}
and
	/.domain-root-write-{name}

we now have

	/.domainroot-{name}
and
	/.domainroot-write-{name}

The (composite) Service and Protocol labels remain unaltered as
	_nfs4._domainroot._tcp
and
	_nfs4._write._domainroot._tcp.

(2) Include an "$ORIGIN example.net." line in the DNS zone data examples
to eliminate one source of ambiguity.

		Craig

---

From: Robert Thurlow <[email-masked]>
Date: Tue, 16 Mar 2010 11:01:32 -0600

> to eliminate one source of ambiguity.

These changes look sane to me, Craig.

Rob T

---

From: Robert Thurlow <[email-masked]>
Date: Tue, 16 Mar 2010 12:13:47 -0600

>   the location of the FSN in a separate file or database.

If the junction does not contain complete information needed
to retrieve FSLs from the NSDB, then a copy/move/backup+restore
of that junction may mean the extra information is no longer
available.  Using extended attributes may be acceptable; we
believe tools should always copy them with their parent object.
An alternate early design had us using our "subfiles" (used for
NFSv4 named attributes) for this.  We did not like the atomicity
issues or the fact that we were stepping on the named attribute
namespace.  We do not have a name/value pair style of extended
attributes readily available.

>   more details on implementations that have limitations on the FSN 

The current Solaris implementation stores junctions in a
specially-marked symbolic link.  We did not increase the
natural size limit of our symbolic links in order to limit
the impact on existing codes that thought they could handle
symlinks, so the total size of our payload is 1024 bytes.
Into that, we could envision that we could be attempting
to store both an NFS referral and an SMB referral, so any
addition to the junction data needs to make sense for us.

To recap some prior discussion:

I step back to thinking that our fundamental need here is
to take a domain and an FSN UUID that is unique to the domain,
and find FSLs for that FSN.  From the domain, I should be able
to find (and cache) the name or address of an NSDB instance,
and find (and cache) the NCE that permits me to find the
correct per-domain info within that NSDB instance.  Storing
the domain plus UUID in the junction thus seems most simple.
I think the admin client will want to do this as well, and
wonder where it can get that information if it cannot do a
similar query.

Rob T

---

From: James Lentini <[email-masked]>
Date: Thu, 18 Mar 2010 10:19:12 -0400 (EDT)

On Tue, 16 Mar 2010, Robert Thurlow wrote:

> > understood it) was that the FSN data (UUID, NSDB's DNS name, and   LDAP
> > search base) would be too large to store in a junction.
> > 
> >   Chuck suggested that the fileserver could store FSN values   separate from
> > the junction. The junction would contain a pointer   to the appropriate FSN.
> > The implementation would be transparent to clients   and not exposed in the
> > Admin protocol. file or database. 
> >
> >   An example implementation would be for junction information to be   stored
> > in an extended attribute. The junction information could indicate   the
> > location of the FSN in a separate file or database.
> 
> available.

This is an important issue to keep in mind. If the file system object 
is unable to store the entire junction information and stores a 
pointer to the junction information instead, it should return entire 
junction information to copy/move/backup+restore operations. An 
approach similar to a sparse file might work, where the junction's 
contents are synthesized from a combination of the file's contents and 
metadata. I suspect that there are other options.

> Using extended attributes may be acceptable; we
> >   We agreed that the best way forward on this topic would be to get   more
> > details on implementations that have limitations on the FSN   size.
> 

Thanks. These are the details we were looking for.
 
> wonder where it can get that information if it cannot do a

This is an interesting though. Do you have thoughts on how this might 
be achieved? How would a domain be mapped to an NCE? Since NCEs are 
defined entirely by the administrator, the mapping would need to be 
performed by a service rather than a static formula. 

-james

---

From: Chuck Lever <[email-masked]>
Date: Thu, 18 Mar 2010 12:10:49 -0400

On 03/16/2010 02:13 PM, Robert Thurlow wrote:
>> Chuck suggested that the fileserver could store FSN values separate
>> from the junction. The junction would contain a pointer to the
>> appropriate FSN. The implementation would be transparent to clients
>> stored in an extended attribute. The junction information could
>> indicate the location of the FSN in a separate file or database.
>

Yeah, hadn't thought of that.

> namespace.

Interesting point.  Are the atomicity requirements for junction creation 
and removal spelled out somewhere?  When creating a junction, is there a 
requirement that the local file server never sees inconsistent junction 
data?

> We do not have a name/value pair style of extended
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

-- 
chuck[dot]lever[at]oracle[dot]com