Subject: Re: [nfsv4] DN contribution to IETF87 meta-discussion

From: "Matt W. Benjamin" <[email-masked]>
Date: Thu, 9 May 2013 16:43:03 -0400 (EDT)

Doesn't it also beg the question, why are implementations incomplete?

I've seen at least one list participant state "I never promised to implement
the entire specification."

So isn't a la carte precisely a solution to the completeness problem--since
if we had a la carte, many of those "incompletions" would have been off the 
menu?

Matt

----- "Nico Williams" <[email-masked]> wrote:
> 
> > Considering that there are no fully complete implementation of
> NFSv4.1 out there, I'm rather surprised that people are so gung-ho
> about wanting to speed development up...
> 
> Completeness of implementations of NFSv4.1 is not really relevant to
> whether some other feature should be adopted or rejected (or even
> denied the individual submission track).  Consider that to go from
> Proposed Standard to Draf Standard, and then to Standard status,
> requires *removing* all those features for which interoperable
> implementation pairs cannot be found.
> 
> Nico
> --

-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 15:52:16 -0500

> Doesn't it also beg the question, why are implementations incomplete?

Yes.

> if we had a la carte, many of those "incompletions" would have been off the

Indeed.  This is a great point.  We already have an a la carte
negotiation mechanism for all those things that are OPTIONAL and where
a peer with support can detect the other's non-support (some optional
features might be matters of semantics that don't require
negotiation).

So the only real issue is about the propriety of mandating version
trains.  Having version trains is fine.  Mandating version trains at
any one moment (that is, now) is OK.  Mandating -now- that version
trains be used in the future is not.

Nico
--

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 9 May 2013 21:23:34 +0000

> if we had a la carte, many of those "incompletions" would have been off the 

We already have optional features for this very reason. The Linux crowd
is not interested in named attributes, so my client doesn't implement
them. By "complete" implementation, I mean one that implements all the
features that are needed to meet the minimal requirements of the spec.

So I still don't see what this proposal is trying to solve. Is the plan
to allow us to add pNFS support to NFSv4.0? That would _raise_ the bar
for implementation, since pNFS has a lot of requirements that would be
much harder to meet without NFSv4.1 features such as reliable callbacks
and sessions.

-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: "Noveck, David" <[email-masked]>
Date: Thu, 9 May 2013 18:06:43 -0400

-----Original Message-----
From: Noveck, David 
Sent: Thursday, May 09, 2013 6:01 PM
To: 'Nico Williams'
Subject: RE: [nfsv4] DN contribution to IETF87 meta-discussion

> So the only real issue is about the propriety of mandating version trains.  

> Having version trains is fine.  

I thought you said it smells.  My presumption was that you didn't mean it smelled vey good.

To put this in terms of a different sense., it doesn't feel right to me.

> Mandating version trains at any one moment (that is, now) is OK.  

> Mandating -now- that version trains be used in the future is not.

Aside from the issue of propriety, has there been any mandate, proper or not, to 
use version trains?    If not, why are people assuming we will be using them and that
not using them is somehow anathema?

So now maybe we should ask those who favor version trains to explain why they are a
good idea.   A treatment of NFS versioning issues should include that perspective as well.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Nico Williams
Sent: Thursday, May 09, 2013 4:52 PM
To: Matt W. Benjamin
Cc: Trond Myklebust; nfsv4 list ([email-masked])
Subject: Re: [nfsv4] DN contribution to IETF87 meta-discussion

Yes.

> I've seen at least one list participant state "I never promised to 
> implement the entire specification."
>
> So isn't a la carte precisely a solution to the completeness 
> problem--since if we had a la carte, many of those "incompletions"
> would have been off the menu?

Indeed.  This is a great point.  We already have an a la carte negotiation mechanism for all those things that are OPTIONAL and where a peer with support can detect the other's non-support (some optional features might be matters of semantics that don't require negotiation).

So the only real issue is about the propriety of mandating version trains.  Having version trains is fine.  Mandating version trains at any one moment (that is, now) is OK.  Mandating -now- that version trains be used in the future is not.

Nico
--
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "Noveck, David" <[email-masked]>
Date: Thu, 9 May 2013 18:07:31 -0400

-----Original Message-----
From: Noveck, David 
Sent: Thursday, May 09, 2013 5:37 PM
To: 'Myklebust, Trond'
Subject: RE: [nfsv4] DN contribution to IETF87 meta-discussion

I think there is a fundamental distinction between clients and servers in this respect.

If a client is not interested in a feature, it doesn't use it.  That applies whether the feature is optional or mandatory.  The distinction is irrelevant in this case.  Except in the case of callbacks nothing can be mandatory for the client and so far, callbacks are not mandatory.

> Is the plan to allow us to add pNFS support to NFSv4.0?
 
No.

> That would _raise_ the bar for implementation, since pNFS has a lot of 
> requirements that would be much harder to meet without NFSv4.1 features such as reliable callbacks and sessions.

Indeed so.   The features you mentioned were made mandatory in v4.1 because they are legitimately infrastructural.  It makes sense to base other features on them.  Even if they were optional, there could well be rules saying that feature A requires feature B.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Myklebust, Trond
Sent: Thursday, May 09, 2013 5:24 PM
To: Matt W. Benjamin
Cc: Nico Williams; nfsv4 list ([email-masked])
Subject: Re: [nfsv4] DN contribution to IETF87 meta-discussion

> would have been off the menu?

We already have optional features for this very reason. The Linux crowd is not interested in named attributes, so my client doesn't implement them. By "complete" implementation, I mean one that implements all the features that are needed to meet the minimal requirements of the spec.

So I still don't see what this proposal is trying to solve. Is the plan to allow us to add pNFS support to NFSv4.0? That would _raise_ the bar for implementation, since pNFS has a lot of requirements that would be much harder to meet without NFSv4.1 features such as reliable callbacks and sessions.

NetApp

---

From: "Myklebust, Trond" <[email-masked]>
Date: Thu, 9 May 2013 23:54:21 +0000

> good idea.   A treatment of NFS versioning issues should include that perspective as well.

The main advantage of version trains as far as I'm concerned is the
control over which features are mandatory and which ones are not.

A la carte is well and good, but as I stated in another email, I would
hate to have to deal with pNFS in a world where sessions are not a
mandatory feature. In fact, I want to deprecate NFSv4.0 as soon as
possible precisely because it _doesn't_ have sessions (and sane
only-once semantics).
A train model gives us a way to decide that henceforth, sessions are
mandatory, and so I can move my customers who are complaining about
mkdir() returning incorrect EEXIST errors to NFSv4.1, secure in the
knowledge that Nico and Chuck can't implement a server called NFSv4.1
which doesn't have correct only-once semantics.

Call it marketing, if you like; the point is that a train model with
mandatory features makes it easy to identify servers that I know will
have those minimal features that I need without requiring me to go to
the marketing departments of Cohortfs, EMC, Microsoft, NetApp, Panasas,
Oracle, Red Hat, and Tonian to find out which of their many server
alternatives does what.

That said, if people want to add optional and non-essential features
outside the train model, I really have no opinion. As long as I can
ignore those features until I find people who care about them, then
fine...

NetApp

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 10 May 2013 00:21:55 +0000

> ignore those features until I find people who care about them, then

One question that I would have about optional and non-essential
features: are we really sure that we need to cram so many of them in a
standards document whether or not we have a train model?

We've racked up quite a few features in the current documents that are
not going to pass those interoperability tests that Nico mentioned.

NetApp

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 20:13:09 -0500

> control over which features are mandatory and which ones are not.

This can be done by publishing an update of the base protocol RFC
indicating which new features (likely defined in other RFCs) are
required to implement and which aren't.

TLS has evolved roughly like that.

Nico
--

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 10 May 2013 01:15:34 +0000

> TLS has evolved roughly like that.

So how does that differ from a train model?

NetApp

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 20:15:38 -0500

> not going to pass those interoperability tests that Nico mentioned.

Yes, that's another thing.  You could have a bare-bones NFSv4.x spec
which you can move to Standard and then have all the extensions (pNFS,
data migration, end-to-end integrity, labeling, ...) be separate so
each can become a Standard whenever it makes sense for it.  And if you
want to make, say, labeling required to implement you then update the
base spec RFC/STD to indicate as much.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 20:17:51 -0500

> So how does that differ from a train model?

Have I misunderstood?  It wouldn't be the first time...  My impression
was that NFSv4 was evolving by having as many features in one minor
version as possible, then the WG focused on that until they were done
and new things mostly didn't get considered until then -- all new
features had to be in a minor version.  I don't mind having an Emily
Litella moment onec in a while.  Did I?

Nico
--

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 10 May 2013 01:21:45 +0000

> base spec RFC/STD to indicate as much.

That would be an interesting idea to explore. That meets the requirement
that we will have core mandatory features in each new version of NFSv4,
and if done right, it should also allow the extensions to depend on
particular features...

NetApp

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 10 May 2013 01:33:23 +0000

> Litella moment onec in a while.  Did I?

OK, so the problem you perceive is that we're currently tying down new
features to a specific minor version, ignoring the fact that those
features might not have dependencies on mandatory features from that
minor version. Is that a correct statement?

So what would be on such a list of "mismatched features"? Are we talking
about things such as copy offload, which could possibly be implemented
against NFSv4.0 if you have safeguards against the callback channel not
existing?

NetApp

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 20:47:20 -0500

Yes.

> against NFSv4.0 if you have safeguards against the callback channel not

For example.  I can think of others:

 - ZFS snapshot / clone / Time Machine type operations.  An ADMIN-type
protocol, but general.

 - end-to-end integrity checking

 - extra cache coherency options (i.e., better cache coherency, a la Lustre)

 - alternate ACL forms

 - who knows?

If it were relatively easy to get whatever bitmask/number/name
assignments then I could see people experimenting with new features
much more easily and, therefore, more often.  If I could write a
kernel->user-land sub-protocol up-caller to build user-land
implementations of ADMIN-type protocols then... I just might.

Nico
--

---

From: "Myklebust, Trond" <[email-masked]>
Date: Fri, 10 May 2013 02:08:28 +0000

> implementations of ADMIN-type protocols then... I just might.

It is easy enough to open the floodgates:

Operation 67: EXPERIMENTAL

ARGUMENT

struct EXPERIMENTAL4args {
	utf8string	identifier;
	opaque	arguments<>;
};

RESULT

union EXPERIMENTAL4res switch (nfsstat4 status) {
case NFS4_OK:
	opaque	results<>;
case NFS4ERR_NOTSUPP:
	void;
};

However it would be nice to avoid turning the IETF into a rubberstamping
board.

We should at least set the bar at writing an RFC, and obtaining rough
consensus from the WG for the design of any new optional feature so that
it is given an official operation (or set of operations). Then have a
process for converting those optional operations into mandatory ones in
new minor versions.

NetApp

---

From: Nico Williams <[email-masked]>
Date: Thu, 9 May 2013 21:28:39 -0500

> Operation 67: EXPERIMENTAL

Putting the word "experimental" or an "x" or "X" affix in the thing
won't do.  We've have lots of bad experiences with that.  The problem
occurs when you decide that something's no longer experimental, and
then you need a flag day or something.

Operation numbers come from a large namespace, no?  All you need is to
make it easier to allocate those (and other things).  By easier I mean
that it should suffice to have Expert Review in the relevant IANA
registry.

Nico
--