Subject: [nfsv4] FedFS Meeting Minutes, 4/1/2010

From: James Lentini <[email-masked]>
Date: Thu, 1 Apr 2010 15:28:04 -0400 (EDT)

FedFS Meeting Minutes, 4/1/2010
-------------------------------

Attendees
---------

Craig Everhart (NetApp) 
Sorin Faibish (EMC)
Chuck Lever (Oracle)
James Lentini (NetApp)
Peter Staubach (EMC)
Chris Stacey (EMC)
Robert Thurlow (Oracle)

Minutes
-------

+ IETF Note Well Agreement

  This is a reminder that our discussions are governed by the 
  IETF Note Well Agreement. See:

    http://www.ietf.org/NOTEWELL.html

  We will start each week's meeting with this announcement.

+ DNS SRV Update

  See draft-ietf-nfsv4-federated-fs-dns-srv-namespace-03
  published on March 30, 2010. 

  The diffs were reviewed previous via email on the WG list.

  Changes are relatively small from -02. 

  The conventional prefix for the export path of a domain was 
  changed from "damain-root" to "domainroot". This matches the 
  spelling in the SRV RR.

  The text now states that locating a domain's root file system 
  SHOULD be performed by a client and MAY be performed by a server.

  For a side-by-side diff with the -02 version, see:
  http://tools.ietf.org/rfcdiff?url2=draft-ietf-nfsv4-federated-fs-dns-srv-namespace-03.txt

  At this time, there are no outstanding issues. Craig will 
  discuss with the WG chairs if this document is ready for WG 
  last call.

  Rob Thurlow suggested that this would be an interesting item to 
  test at the June Bake-a-thon.

  Peter Staubach said he has been working on a design for a server-side 
  implementation.

+ Admin Protocol: Junction Size

  James configured an OpenLDAP server with a FedFS 
  attribute in the root DSE. For instructions on how to 
  set this up, see this message to the mailing list:

  http://www.ietf.org/mail-archive/web/nfsv4/current/msg07800.html

  Rob Thurlow will check on how to configure this in OpenDS.

  Rob Thurlow asked about the junction could store a domain name 
  rather than a DNS name. We reviewed the lack of a way to map a 
  domain name to a set of LDAP directories. We noted that a 
  DNS name could be configured to map to multiple IP addresses and 
  hence this option supports the same configurations.

  James Lentini suggested that if adding an attribute to the root DSE is 
  problematic, then other options could be considered. One alternative 
  might be for the root entry of each of the root DSE's namingContexts 
  to contain the FedFS NSDB Container Entry pointer.

  James Lentini searched the LDAP specifications for text requiring directory 
  implementations to allow for modifications to their root DSE. The 
  most relevant text he found was:

   Servers may allow clients to modify attributes of the root DSE, where
   appropriate.

  on page 37 of RFC4512. This text doesn't say what the server administrator 
  is allowed to modify.

+ Admin Protocol: International Strings

  Chuck Lever summarized the international domain name (IDN) discussion 
  on the WG list. There appears to be confusion about the use of IDNs 
  in RFC3530 and RFC5661. There is a request for clarifications to be included in 
  the 3530bis draft. 

  Rob Thurlow noted that clarifying text is allowed to be added in a bis document, 
  like 3530bis, and could also be logged as errata against an RFC, like 
  RFC5661. 

  Our plan was for the FedFS Admin protocol to reference the conventions for 
  international strings used by NFSv4.0/4.1.

  Chuck Lever shared his perspective as a designer of a FedFS admin client 
  and server. He doesn't have a preference for a particular format. His main 
  concern is that both ends have an agreed upon format. He simply wants a 
  defined, programmatic way to determine what to do with strings in the protocol.

  Chuck observed that punycode is used on the wire by many other protocols.

  [AI] James Lentini will note dependency on NFSv4.* policy.

+ Admin Protocol: Pathnames

  We continued our discussion from the WG email list.

  Chuck Lever is concerned about the implementation complexity.

  Chuck Lever tried having fedfsd read /etc/exports on Linux. The 
  format of /etc/exports is pretty complicated (FSIDs, fs_locations, etc.). 
  He doesn't believe that this approach is feasible.

  He is looking into a user-kernel API to convert a path to a file descriptor.
  This appears to be hard, but not insurmountable.

  Would a kernel implementation of the admin protocol be better? 
  It is something to consider. It would solve some problems, but raise others. 

  Chuck Lever suggested that making the Admin protocol part of the core NFS 
  protocol would solve this problem. We noted that our official charter was 
  to not modify the existing NFS protocol. Such a proposal could be made in 
  a future NFS version.

  Chuck Lever suggested an alternative way to specify the path structure. 
  Instead of a struct with a tag and path:

  struct foo {
    tag
    path
  }

  a discriminated union would allow for each tag type to define its own 
  path type:

  union foo (tag) {
   TAG1:
      path1
   TAG2:
      path2
  }

  We agreed that a union is better than a struct because it allows each 
  type to specify its own path types (some might not be an array of strings).

+ Next Meeting

  Our next call will be in two weeks on April 15.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2010 15:44:24 -0500

>   defined, programmatic way to determine what to do with strings in the protocol.

If we declare our domainname slots IDN-aware then receivers have to be
prepared to handle A-labels and U-labels both.

>   Chuck observed that punycode is used on the wire by many other protocols.

That doesn't mean we must.  See the thread on this on the list.

>   to not modify the existing NFS protocol. Such a proposal could be made in 

Parts of NFSv4 implementations in Linux and Solaris (and *BSD?) involve
user-land today via "upcalls".  GSS-API security context token exchanges
(via gssd) and user/groupname@domain resolution (via nfsmapid) come to
mind.

I would recommend an in-kernel ADMIN server implementation with upcalls
to a user-land helper for those parts that belong squarely in user-land
(e.g., DNS resolution, LDAP).

>   [...]

Indeed, this is better.

Nico
--

---

From: Chuck Lever <[email-masked]>
Date: Thu, 01 Apr 2010 17:28:52 -0400

>>    This appears to be hard, but not insurmountable.

Such an API would utilize current mountd upcalls to help resolve 
incoming NFS pathnames.  Some other mechanism would be needed for 
handling pathnames for other file protocols.

> (e.g., DNS resolution, LDAP).

Not all file server implementations allow other local accessors. 
Perhaps the most general approach would be for fedfsd to operate on 
junctions within a server's exports via the server's own file access 
protocol, and not via direct local access of junction objects.  That 
architecture could work for all styles of file servers, and might ensure 
that our ADMIN protocol design does not exclude some implementation types.

For an NFS-only prototype, however, we might consider one of the 
solutions suggested above.

-- 
chuck[dot]lever[at]oracle[dot]com

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2010 16:45:14 -0500

> Not all file server implementations allow other local accessors.

This, IMO, is the best argument for running the ADMIN protocol on port
2049, alongside NFSv4.x (and v3).  Yes, this means that making the ADMIN
server work for CIFS becomes much harder.

But arguably making the ADMIN protocol work for such implementations is
the implementor's problem (in your case that means that you'd have to
get buy-in from the Linux NFS server maintainers; for someone else it
might mean getting another implementor to provide some kind of local
access API, even if private).  And making the ADMIN protocol extensible
to protocols other than NFS is highly desirable.

I'm not convinced that we need to do anything more than make pathnames a
tuple:

    {access protocol,
     access protocol instance (port number),
     <access-protocol-specifici-path>}

In XDR that'd be a discriminated union with the file access protocol as
the discriminant and, for NFS paths, a port number and an NFS-specific
path type.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2010 16:51:16 -0500

>      <access-protocol-specifici-path>}

Also, an implementor who has to interface between multiple distinct file
access protocol implementations might want to make their ADMIN protocol
a proxy for local ADMIN instances (as many as one per-file access
protocol instance).

---

From: Chuck Lever <[email-masked]>
Date: Fri, 02 Apr 2010 13:14:51 -0400

> server work for CIFS becomes much harder.

I don't think it argues for such a specific solution.  It is a good 
argument for providing junction management via a file server's front 
door, and the example above is just one way to implement it.

An auxiliary admin protocol flies in the face of the current NFS trend 
to avoid auxiliary protocols.  However, the FedFS charter prevents us 
from considering changes to the file access protocols for the time being.

Perhaps the admin protocol draft would be improved by including the 
suggestion above along with some discussion about its strengths and 
limitations.

> to protocols other than NFS is highly desirable.

Perhaps we should focus on including other implementers in the protocol 
design discussion.

> the discriminant and, for NFS paths, a port number and an NFS-specific

With this design, we have more flexibility in specifying the location of 
putative junctions.  CIFS, for instance, would want a way to specify the 
share name to go with the path.  We might also define a path type for 
specifying an NFS file handle instead of a pathname.

It might be useful to include an access protocol enum value (say, 
"local", and the path type would exclude the port number) that allows 
the admin client to specify absolute local pathnames on the server.

That would offer the same functionality as in the current draft 
language, but would be optional.  You could make it a "SHOULD support" 
rather than a "MUST support".  If an implementer can't provide this 
support (if for example, the file server doesn't allow local access to 
its file storage), or such pathnames prove to be a security issue, 
access via this type of pathname can be disabled, while still allowing 
some ability to manage junctions on the server.

This would define a simple pathway for some implementers to easily 
support the admin protocol without worrying about remapping exported 
pathnames to local pathnames, and without excluding implementations that 
provide no local access to their local file store.

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 2 Apr 2010 12:22:48 -0500

On Fri, Apr 02, 2010 at 01:14:51PM -0400, Chuck Lever wrote:
> An auxiliary admin protocol flies in the face of the current NFS
> trend to avoid auxiliary protocols.  However, the FedFS charter
> prevents us from considering changes to the file access protocols
> for the time being.

Agreed.  My suspicion is that the auxiliary protocol approach derives in
part from the difficulty in extending the base protocol.

> Perhaps we should focus on including other implementers in the
> protocol design discussion.

Sure, but we might not finish anytime soon (not an argument to exclude
anyone, nor to not give others a heads-up, but to not wait for them).

> With this design, we have more flexibility in specifying the
> location of putative junctions.  CIFS, for instance, would want a
> way to specify the share name to go with the path.  We might also
> define a path type for specifying an NFS file handle instead of a
> pathname.
> 
> "local", and the path type would exclude the port number) that
> allows the admin client to specify absolute local pathnames on the
> server.

Sure.

> pathnames to local pathnames, and without excluding implementations
> that provide no local access to their local file store.

We agree.  Do we now have consensus on a way forward?

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 2 Apr 2010 12:24:41 -0500

> We agree.  Do we now have consensus on a way forward?

To clarify, I meant "Chuck and I" by "we", not some other set of people.

---

From: Chuck Lever <[email-masked]>
Date: Fri, 02 Apr 2010 13:26:59 -0400

> We agree.  Do we now have consensus on a way forward?

This is OK with me.  I think we can confirm this, with any follow-up 
discussion, during the next phone call on the 15th.

---

From: "Mike Eisler" <[email-masked]>
Date: Mon, 5 Apr 2010 17:08:38 -0700

> from considering changes to the file access protocols for the time being

Leaving aside the issue of goal of making FedFS as file-access-protocol-agnostic
as possible, NFSv4 contains all the operations for accessing files 
not adminstering them.

We don't, for example, include operations for exporting path names or setting
quotas.

An auxiliary admin protocol should not be part of the NFSv4.x file access
protocol. The main reason it is ONC RPC-based is because the
WG's expertise in client/server protocols is primarily in ONC RPC;
WG implementers can get to products quicker with RPC.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Chuck Lever
> Sent: Friday, April 02, 2010 10:15 AM
> To: Nicolas Williams
> Cc: [email-masked]
> Subject: Re: [nfsv4] FedFS Meeting Minutes, 4/1/2010
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "Everhart, Craig" <[email-masked]>
Date: Tue, 6 Apr 2010 10:45:33 -0400

There are a couple somewhat separate issues here.

(a) Administering a shared multi-server namespace.  NFSv4 with referrals
lets multiple servers project a uniform namespace that customers may
well want to administer in terms of that projected uniform namespace.
Adding create-junction and delete-junction operations to NFSv4 would
enable that administration directly.  (We already have getattr-junction
because the client sees the "fs_location*" attributes.)

(b) The existing admin protocol proposes creation/deletion of junctions
by a server-specific pathname, which makes sense as part of the
individual administration of servers.  But a single server alone cannot
interpret paths from that projected uniform namespace.

There are a variety of architectures that might be used for a hybrid
approach.  One is that the namespace administration tool might learn
some things from the NFSv4 client, like the server and path of the
innermost file system mounted, and the path relative to the root of that
file system.  The admin tool then takes all that information and does
operations with the file server's admin server.  Alternatively, the
admin tool could learn NFS filehandles from the NFSv4 client and
communicate them to the admin server.  This opens the questions about
the volatility of the filehandles and so forth.

It's really tempting, instead, to try to leverage the NFSv4 client and
server protocol for this.  The NFSv4 client is already dealing with all
these issues and clearly would be operating on exactly the projected
uniform namespace.

		Craig

> From: Mike Eisler [mailto:[email-masked]]
> Sent: Monday, April 05, 2010 8:09 PM
> To: 'Chuck Lever'; 'Nicolas Williams'
> Cc: [email-masked]
> trend
> trend
location
> of
allows
> that