Subject: [nfsv4] Minutes from NFSv4 WG Meetings at IETF65

From: Lisa Week <[email-masked]>
Date: Wed, 29 Mar 2006 18:31:46 -0700

NFSv4 WG Meetings
March 22 and 23, 2006
65th IETF - Dallas, TX
Note: List of action items and issues is at the end of the minutes.

Wednesday, March 22, 2006
=================

Welcome / sign blue sheets / notewell

Agenda:
--------
* Message from new area directors  (Lars Eggert)
* Connecathon update (TBD)
* Linux pNFS experiences (Goodson/Eshel)
* DNS Namespace (Everhart)
* ACL update and plan (Shepler)
* NFSv4.1 minor version (Shepler)

Agenda additions:
------------------
None mentioned

Message from the new Area Directors (Lars / Magnus)
----------------------------------------------------
Area directors will be working with WG to figure out what is going on.  
Lars noted that either of the Area Directors are are storage guys.  He 
also noted that there is nothing on "fire", which is good.  Lars asked 
about getting milestones updated for the WG.  One of the major 
milestones is to get the group's charter up to date.

Issue raised by Beepy was that the NFSv4 WG has been trying to get 
charter up to date on website, but nothing ever happens.  Charter has 
been approved, but just needs to get posted.  Beepy to send ADs the 
wording that is supposed to go on website.

Spencer et al. went into a summary of what the WG has been up to and 
what its charter includes.  Items mentioned/discussed are as follows:
* XDR protocol went to full standard.

* Base level RPC protocol ready to move to draft standard.  Most 
contention is around registration process of program names.  Need to 
move registration process to IANA.  If we move NFSv4 to draft, we have a 
dependency on the RPC status, therefore, the registration registration 
process should be addressed at some point.

* RPC over RDMA - RPC/RDMA problem statement and 2 protocol docs in WG 
status at this point.  There haven't been any comments for a year and 
Tom Talpey stated that he thinks protocol is done.  Three 
implementations already in place and 2 more being worked on.  At the 
moment, it is deemed more appropriate to work on 4.1.

Discussion will be taken offline in order to determine 
timing/milestones.  Tom to get any last comments on RPC/RDMA docs.

* SPKM - Existing SPKM3 Implementations: Linux/CITI and IBM
As the Linux/CITI implementation of SPKM3 went forward, they found that 
the SPKM3 doc was underspecified.  With that being the case, the 
implementation was difficult.  Therefore, a new I-D combining RFC2847 
with some clarifying text was submitted.  It is thought that this should 
be a standards track document.

The question of who owns the document was raised.  The I-D was sent to 
the NFSv4 WG.  It was also sent to kitten, but they didn't feel that 
they should own the issue because they don't do mechanisms.

Going forward, it is thought that it may be best to work the issue in 
the Security WG.  Andy to get I-D resubmitted and to work with Spencer 
on closing the issue of who owns the document.

* Server to server replication/migration protocol - There is still some 
interest in pursuing this.  Dave Noveck has proposed possibility of 
using v4 with additions to do server to server migration, but has stated 
that he doesn't know where we really are with that and with 4.1 he 
doesn't know how far to push the issue.

 From charter standpoint, we need to work forward with requirements and 
validate need for this.  Dave Noveck to make 3rd section of original I-D 
a stand alone document.

* Namespace - Help identify clients and servers.  Been working with DNS 
WG on the NFS4 ID record.

* NFSv4.1 (minor version) - WG is working on regenerating large document 
which includes all of 4.0 and all of 4.1.  The idea is to have a stand 
alone document with section on general content (this section hasn't been 
written yet, but will be in the next draft).  Spencer recommended that 
the ADs wait until the next version (-03) of the draft before looking at 
doc in any detail.

David Black recommended putting a "guide for the uninitiated" in the 4.1 
doc.  Alternate approaches: side doc describing 4.1

Connectathon Update:
---------------------
Testing included NFSv4 as well as pNFS.

NFSv4:
Testing was fairly uneventful.  Lots of different clients and servers. 
Not a whole lot was done with additional features (i.e. no 
replication/migration).  Meetings were held on ACLs and pNFS.
For summary of the ACL meeting see: 
http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html
For minutes from the pNFS meeting see: 
http://wiki.linux-nfs.org/index.php/Cthon06_Meeting_Notes

pNFS:
Client implemententations tested: Linux client which is being worked on 
by IBM and CITI.
Server implementations tested: NetApp and Dean Hildebrand's pvfs server.

Successful in mounting the NetApp filer using Linux client which supports
both files and pvfs layout drivers.  CITI is working with IBM to get
another files based server up and running, but this wasn't at Connectathon.

Question on whether any protocol or implementation changes came out of
pNFS testing was asked.  The following changes, not necessarily due to
testing, were mentioned:
1.) May need new attribute that will tell the client what i/o's should
go to md server and not bother getting a layout.
2.) LAYOUTCOMMIT: issue whether you need to keep writes around until the 
LAYOUTCOMMIT.  Be nice to free pages once data has been written to the 
data servers.  This issue differs per layout type.  This problem exists
in the block world.  WOuld be nice to generalize.  Would the RECLAIM 
stuff cover the files case?

Question on whether or not anyone has done any performance work to 
figure out if we are going to run into problems in the future.  General 
answer was no.

Do we need implementation guide/draft which states certain design 
characteristics need to be used due to protocol requirements.  Worry is 
that if we don't get some of these things written down we will be going
wasting time discussing and solving the same problems multiple times. 
Consensus was that this was a good thing, but it was more of an issue of 
who really has time to do it.

The idea of creating a section off of the nfsv4-editor.org site to hold 
issues specifically about the design considerations portion of pNFS was 
raised.  No action items created.

DNS Namespace (Craig Everhart):
------------------------------
Initial goal is to make the mapping for domain name to file system exist 
in the first place.  Wants to solve the "how do I configure my client 
quickly" problem.

Proposal to use DNS SRV RR to map from domain names to file system 
roots.  Populate the second level of a global uniform name space with 
domain names on domain.  Populate the third level as those mapped
to file system roots.

Relationship to NFSv4 and it's subtleties:
Need convention for root: /nfs4/example.net
Convention for r/w: /nfs4/.example.net
No further mount options
Choice of port in DNS SRV RR

Organizations can use this as a conventional way to publish certain file 
systems or different types of information.
Question of whether this should be done in server or in client was 
raised. Client seems to be preferred but not necessary in principle.

This draft was run by the DNS wg.

Issue of the port number not mentioned in the referral text was raised.  
There wasn't consensus on whether this is a problem or not.

Question on how do you publish a file system for a domain name was 
asked.  Craig doesn't believe the decision needs to be made as far as to 
where to put this capability.  He believes it is an implementation 
detail as to whether it is client or server issue.

Question on whether or not there is a convention needed for clients to 
know which domain to connect to.  Perhaps based on the domain of the 
client.

What are the future paths?  Come up with recommendations on using this
for purposes other than occasional discovery.

Going forward, if v4 WG is okay with the proposal, get review from DNS 
group.

ACL Issues (Sam Falkner):
-------------------------
Discussion at cthon revolved around windows interoperability, 
POSIX-conformance and POSIX-draft mapping.
See the following email for a summary of the meeting:
http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html

Consensus was that multiple documents are needed, all of which should be 
Informational RFCs.

1.) Bruce Fields to create a document describing Windows Interoperability.
This document should capture info like what really happens on Windows to 
accomplish dynamic inheritance (i.e. the issues that were brought up in 
response to the above email summarizing the Connectathon ACL meeting).

2.) Bruce Fields to revive the POSIX-draft ACL to NFSv4 ACL mapping doc.

3.) Lisa Week and Sam Falkner to create document about options for
POSIX-conformance in an environment with NFSv4 ACLs.  This document will 
include the POSIX Considerations section from 
http://www.ietf.org/internet-drafts/draft-falkner-nfsv4-acls-00.txt

NFSv4.1 Update (Shepler):
--------------------------
NFSv4.1 I-D is at *-02.

Spencer has created http://www.nfsv4-editor.org to help manage the 
issues with the NFSv4.1 doc.

The source for the NFSv4.1 doc has been split up into separate XML 
files.  Spencer to to publish XML files as well as HTML source for 
document.

Moving forward:
I-D is NFSv4.1 only because consensus was that it should be 4.1 only.

Question on how should we move the protocols forward to draft standard 
was raised.  Things to consider: Can we move either 4.0 or 4.1 to draft 
standard without the other?  Lars noted that if left as separate RFCs, 
it is okay to move one forward without the other.

Question on how should we clean up the 4.0 doc based on the 
interpretations that have been done on the WG alias was asked. A couple 
of options were mentioned by the ADs:
1.) Create one document describing 4.0 that obsoletes RFC3530 then have 
a 4.1 be a separate doc.
2.) Create errata for RFC3530

Consensus was to continue to have 2 docs.  Rationale being that keeping 
them separate allows 4.0 to be moved to draft standard.  If there is a 
strong need to modify 4.0, we could create a bis document.  Bissing 
RFC3530 could be done in order to update references such as SPKM3 and 
bring the doc up to date.

Since the docs could get pretty lengthy, it was noted that there may be 
difficulty in moving documents through IETF with large amount of pages.
As a way to move things forward better Lars recommended:
If there are large portions of the two documents that are the same, add 
a section stating which sections are the same in order for others to be 
able to review the document knowing which portions are the same.

The WG is requiring implementation experience before letting the 4.1 doc 
out of wg last call.  Consensus is that we need strong implementation 
experience.

Should we start tracking who is doing what?  How should this be done?  
Doesn't have to be done today, need to accomplish this moving forward.

What is the timeline for the doc?  When do people want a completed 
document and when can we achieve implementations?  In response, the 
question of when do we want to get the doc to the state where people can 
do complete prototypes was asked.  To do that effectively, the doc 
really needs procedure numbers, error numbers, etc.  In order to do this 
we should have the .x file which generally reflects the doc.

What date do we want the .x file by?
Spencer wants reasonable dates.  We'll close tomorrow out with setting 
milestones and goals.

CITI willing to do a Bakeathon in September, but the date is not yet 
set.  Question on what the focus of Bakeathon should be was asked.  
Should it be 4.1 or 4.0?  In response to this it is thought that there 
may need more up front planning for this particular event...

Major Items Missing in -02:
* Introduction section
* RPC/security flavors/transports
    - Shepler to make a pass at this section and send out for review.
* Move Sessions to RPC section
    - Sessions work needs clean-up in order to have more     alignment 
with sessions and pNFS concurrent operations.
* Migration / replication / namespace
    - Dave Noveck to rework the replication/migration section.
* Re-write of lock_owner/open_owner use (Need to rewrite state section to
make sure we are using the correct terms in the correct areas.  Shepler
plans to generalize them by possibly calling it the "owner" and only use 
lock/open in appropriate places.)
* RECLAIM_COMPLETE
    - Noveck to send Shepler text and Shepler to put text into the 4.1 
    draft.
* Include SPKM update?

Tom Talpey brought up the issue about not having made the decision on 
whether or not sessions is mandatory for not.

Feedback on concalls:
---------------------
Definitely useful.  The frequency and length of meetings was good.
Next concall schedule should be every other week, but kept at hour 
duration.

Shepler to publish -03 doc in next week or so.

General pNFS discussion:
------------------------
In order to optimize performance of implementations with cluster fs back 
ends, Mark Eshel is proposing a way to distinguish between data commit 
and metadata commit.

Need for clarifications between data commit, layout commit and file commit
clarifications was brought up.  Goodson to write up the current status 
of the spec.

Question of what if data servers and metadata servers have different 
security mechanisms in use was asked.  Consensus was that there needs to 
be some text to address this.  Spencer to create an issue for this.

Thursday, March 23, 2006
========================

A pNFS specific meeting was held on Thursday morning.  Brent Welch to 
send out minutes from the meeting.

Short summary of what was discussed in Thursday morning pNFS meeting:

The LAYOUTCOMMIT/COMMIT issue regarding cluster file system back ends 
was discussed.  Consensus is that it is okay to modify LAYOUTGET to 
serve the cluster file system back end if it doesn't disrupt the general 
protocol, but a more detailed proposal is needed.  Mark Eshel to write 
up proposal.

What to do for the Bakeathon was also discussed at the separate pNFS 
meeting.  Items mentioned: Directory delegations, pNFS, sessions 
(Linux), secinfo, fs_locations

pNFS EOF issues were discussed. (i.e. If there are layouts outstanding 
and a SETATTR comes in that lands right in the middle of the layout.  
Server shouldn't respond to SETATTR until the layouts are returned.)
David Black to put update on EOF issues into the issues tracker.  Text 
needs to be written.

Open Discussion:
================

pNFS:
-----
Garth has a list of small stuff related to pNFS implementation details 
that hasn't sent to the WG alias.

Issue on when you do LAYOUTCOMMIT: Trond has an outstanding AI to 
summarize this for the WG alias.  (This is more of a best practices for 
the client than related to the protocol)

Question on whether or not there is implementation work going on w/ 
Object and Block based pNFS.  Response was yes.  EMC working w/ CITI on 
blocks driver.  Panasas working on Objects.  Sun working on Objects (and 
Files).

LAYOUTRECALL path is next, we'll be learning about that next.

Whether or not to require Sessions:
-----------------------------------
In the case of pNFS, the consensus is that it requires on sessions.

We need to decide whether 4.1 (not only pNFS) requires Sessions.  (i.e. 
If you announce 4.1 as a version number, are you announcing sessions 
support?)  All interesting features of 4.1 require sessions, so why not 
just require them?  There have been some dissenting opinions w/ regard 
to requiring Sessions for directory delegations.

Benefits of making Sessions mandatory:
* Has potential of improving service all together.
* If we don't have sessions, we'll have to implement work arounds to 
make up for them.  This may be a lot of duplicate work.
* Sessions makes it possible to have callbacks on the same channel.
* NFSv4.1 is a better protocol with Sessions.  Improves delivery of what 
we can provide to customers.

Counter argument:
There are some things in 4.1 that are useful without Sessions such as 
new SECINFO operations.

Sessions Implementation experience:
Implementations of Sessions have been done on Linux NFS server and 
NetApp filer.  Tom Talpey stated that the implementations are relatively 
easy and changes are well contained.  It is not necessary to implement 
the full sessions facility to get the benefit and implementation 
experience shows that it is relatively easy to implement something that 
gives you the benefits of sessions.

Discussion on which portions of Sessions are required:
It is possible that only portions of the Sessions functionalities may be 
mandatory if you advertise 4.1.  The basic required functionality must 
be determined and written up.

Implementations are really just required to support the sessions 
protocol.  They don't really need to support failover, trunking, 
multipathing.

Tom to write statement and back up for making sessions required.

Consensus was that the issue of whether or not directory delegations 
require sessions will be put on hold until the above issue of whether or 
not NFSv4.1 requires Sessions is closed.

General issues:
------------------
Currently, there is no info on how the client recovers after the loss of 
a Session.  Need explanatory text for client recovery when sessions are 
enforced.  Talking about how slot table resource management is handled.  
Tom to add issue to tracker.

Ops w/ large idempotent response along with non-idempotent response.  Is 
NFS4ERR_RESOURCE right or not?  Text may need to be changed.  Eisler to 
add issue for this.

In 4.0 people are using NFS4ERR_RESOURCE and NFS4ERR_DELAY in 
interchangeable ways.  Need to clarify what the errors mean so that we 
all use them in consistent ways.  Eisler to open issue.

Issue on which principle can do which operations after open was raised.  
Need to make sure principle matches stateid before doing reads/writes.  
Eisler has laid out the security/denial of service considerations out in 
a past email.  He will take responsibility of driving the issue to closure.

Question on whether NFSv4.0 has gotten in the way of or has helped out 
with HA-NFS was asked.  If anyone has experience, is there anything that 
should be changed to better support this?  It was mentioned that Bruce 
Fields brought up the fact that the nfsclientid4, as currently 
specified, includes the server IP address and since this will be 
different in a cluster things get difficult.  Spencer mentioned that the 
reason the IP address is there is in order to differentiate between 
server interfaces. There needs to be an nfsserverid4. This is considered 
to be under the umbrella of Sessions and an issue will be captured by 
Dave Noveck.

Question on what the status of Bruce Field's proposal to allow for a 
client to cancel the lock upon recovery was asked.  The group recalled 
that consensus was that this was a good idea (i.e. nothing bad was said 
about it).  Andy to create an issue for Bruce drive this to closure.

Question of where we are on the issue of recovering delegations was 
asked  Consensus is that we have to do something about it.  Eisler to 
write proposal.

Andy mentioned that David Richter (working on directory delegations at 
CITI) has some directory delegations issues to raise.  Andy to ask David 
Richter to put issues in tracker.

There are still open issues regarding fs_locations, but Dave Noveck will 
include the current proposal in the associated text he will provide to 
Spencer for the 4.1 doc.  Even though the text is to be included in the 
4.1 doc, the issue will be left open for discussion.

David Black mentioned that he believed that we need a section added to 
the 4.1 draft to explain what is required for an implementer to 
implement 4.1.

Issue of arbitrarily large compounds was mentioned.  Andy proposed that 
we should set up a list of standard compounds that the server should be 
able to accept.  This would allow the client to know which compounds are 
always going to work with any server's implementation.  Consenting 
opinion was that the server could just return NFS4ERR_RESOURCE and the 
client should modify the compound accordingly.

What is the timeline for Block and Objects RFCs?  David Black's 
intention is to have the Blocks I-D to parallel the 4.1 draft.  Brent 
asked what the process was for the Objects draft to turn to RFC.  
Answer: Documents switch from I-D to RFC only upon going through WG last 
call, iesg review, etc.

Issue on how clients can determine what the capabilities of the server 
are was mentioned.  The running consensus is that in order for clients 
to discover the capabilities of the server, just try the operations.  If 
it fails, it doesn't support it.  If it succeeds, it does.  
Implementation IDs were also mentioned in regard to whether or not they 
could be used for this.

Spencer needs to fix the "Minior" version misspelling in the 4.1 I-D.

utf8/symlink issues:
--------------------
There have been complaints against 4.0 because it didn't specify a 
normalization form.  The problem is thought to be that there are 
multiple ways to represent the same string in utf8 (superscript, etc.), 
therefore we need to normalize the use.  Individuals in the meeting 
don't agree that there is a problem.  Noveck stated that he doesn't 
think things should be changed unless we're trying to solve a real 
customer problem.

It is thought that stringprep functionality couldn't live in NFS, but 
rather in the client above NFS so it does a common thing for NFS as well 
as all other storage protocols.  Even though it is a little different 
than in the case of NFS, David Black mentioned that iSCSI uses 
stringprep but it is above the level of the protocol.

This issue is considered to be closed.  No change is thought to be 
needed at this time.  Even though the issue is considered to be closed, 
David Black warned that IESG will be looking for this so we need to be 
prepared to handle the inquiries.

Schedule discussion:
--------------------
Next IETF meeting is July 9-14, 2006 in Montreal and I-D deadline is on 
June 26.  Spencer to have the 4.1 I-D into the queue the Monday before.

WG will hold conference calls every two weeks until IETF meeting.
These will be during the weeks of: 4/3, 4/17, 5/1, 5/15, 5/29, 6/12

Conference call will be rescheduled to 10am CST and will stay at a 
duration of 1 hr.  Rough consensus in room is to have meetings on 
Tuesdays.  Spencer to send out email to WG in order to finalize the 
meeting day.

Summary of Action Items:
================
AI: Get charter up to date

AI: Beepy to send charter wording to Area Directors for posting on web.

AI: Tom Talpey to poll the WG for any last comments on the RPC/RDMA drafts.

AI: Bruce Fields to create a document describing NFSv4 ACL and Windows 
ACL Interoperability.  This document should capture info like what 
really happens on Windows to accomplish dynamic inheritance.  See the 
following email thread: 
http://www1.ietf.org/mail-archive/web/nfsv4/current/msg02792.html

AI: Bruce Fields to revive the POSIX-draft ACL to NFSv4 ACL mapping doc.

AI: Lisa Week and Sam Falkner to create document about options for 
POSIX-conformance in an environment with NFSv4 ACLs.

Issues to be added to tracker (iff they haven't been added already):
=======================================
Issue: Andy Adamson to work with Spencer to determine who owns the SPKM3 
I-D going forward.

Issue: Andy Adamson to send out update to the SPKM3 I-D.

Issue: Dave Noveck to separate section of the Replication/Migration I-D 
explaining the server to server replication/migration protocol into a 
stand-alone document.

Issue: Spencer to publish HTML version and XML files making up the 4.1 doc.

Issue: Spencer to write the text for the RPC/Security Flavors/Transports 
section of the 4.1 doc and send out for review.

Issue: Dave Noveck to rework the Replication/Migration section of the 
4.1 draft.

Issue: Spencer to clarify areas of the 4.1 doc that refer to lock/open 
owner.

Issue: Shepler to include the RECLAIM_COMPLETE proposal into the 4.1 doc.

Issue: Dave Noveck to send Shepler RECLAIM_COMPLETE proposal text.

Issue: Spencer to publish -03 version of the 4.1 doc.

Issue: Spencer to create issue for the creation of text for the pNFS 
problem of what if data servers and metadata servers have different 
security mechanisms in use.

Issue: Mark Eshel to write up a proposal for the solution of the 
LAYOUTCOMMIT/COMMIT issue regarding cluster file system back ends.

Issue: David Black to create issue in issue tracker for pNFS EOF issues 
that were discussed at the pNFS meeting.

Issue: Garth Goodson to send out list of pNFS related implementation 
details.

Issue: Tom Talpey to write proposal for making Sessions required by 
NFSv4.1.

Issue: Tom Talpey to add issue to issue tracker for the creation of 
explanatory text for client recovery when sessions are enforced.

Issue: Andy Adamson to create issue in tracker for reviving Bruce's 
proposal for a client to be able to cancel the lock upon recovery.

Issue: Mike Eisler to write proposal for recovering delegations.

Issue: David Richter to put directory delegations issues in tracker.

Issue: Mike Eisler to add issue to tracker for the need to clarify the 
meaning of NFS4ERR_DELAY and NFS4ERR_RESOURCE errors.

Issue: Mike Eisler to drive issue related to which principles can be 
used for what operations after open to closure.

Issue: Dave Noveck to capture issue on whether there needs to be an 
nfsserverid4 or not.

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4