Subject: [nfsv4] proposed text for v4.2-based conversion to a more incremental protocol extension model

From: David Noveck <[email-masked]>
Date: Fri, 3 Oct 2014 08:53:45 -0400

I've attached a new section to be added, if people are OK with it, to the
NFSv4.2 spec.  The idea is that, if we want to move to a less monolithic
development model, we should do it as soon as possible.  We had  been
thinking that "as soon as possible" meant on the next minor version to
start out (i.e. v4.3), but i think we could do this in the next minor
version to go to IETF last call, namely NFSv4.2.  If that seems too abrupt
for people we could still use the same sort of scheme in v4.3, but if we
do, we should do that with the  understanding that v4.3 would be a
micro-version, consisting of extensibility and not much else (I think
xattrs are far enough along to merit inclusion but others might not
agree).  That might enable us to have a small but extensible v4.3 out
within a year.

This is not within Tom's three-page limit.  It is about four pages and
would allow the v4.2 spec to easily stay under 100 pages, as Tom thinks
important (and so do I).

As Tom has pointed out, there is a lot of interest in a less monolithic,
more running-code-oriented protocol development process.   I think this is
easier than many people believe because the NFSv4 XDR extension model gives
us the infrastructure we need.   I think the issue is not the level of
working group energy, but a general sense of uncertainty about what is
possible here, plus the press of other business (i.e. this is not anybody's
full-time job).

I think that if people realize that this can be done and the working group
decides that this is important, we can effect the necessary transition.

I appreciate people''s comments and suggestions.  However,  I will be
reluctant to try Tom's patience by adding much more material when I'm
already over three pages.   In the context of a v4.3-based proposal, we
have more flexibility, but I'd hope we can  keep any spec on a new
extension model down to 20-25 pages.

---

From: Marc Eshel <[email-masked]>
Date: Fri, 3 Oct 2014 11:05:10 -0700

I would vote to add it to NFSv4.2 as long as it will not delay starting to 
work on other future items like this document proposes. There are items in 
NFSv4.2 that don't have even one reference implementation. I implemented  
NFS4_OP_ALLOCATE, NFS4_OP_DEALLOCATE,  NFS4_OP_IO_ADVISE, 
NFS4_OP_READ_PLUS,  and NFS4_OP_SEEK on Ganesha and tested some of them 
with what Anna did on the Linux client side. I don't know of any work on 
the other operations. I think we should have at least one implementation 
of each new OP.
 
 Marc.

From:   David Noveck <[email-masked]>
To:     "[email-masked]" <[email-masked]>, 
Date:   10/03/2014 05:56 AM
Subject:        [nfsv4] proposed text for v4.2-based conversion to a more 
incremental protocol extension model
Sent by:        "nfsv4" <[email-masked]>

As Tom has pointed out, there is a lot of interest in a less monolithic, 
more running-code-oriented protocol development process.   I think this is 
easier than many people believe because the NFSv4 XDR extension model 
gives us the infrastructure we need.   I think the issue is not the level 
of working group energy, but a general sense of uncertainty about what is 
possible here, plus the press of other business (i.e. this is not 
anybody's full-time job).

[attachment "v4.2PlusOmega.txt" deleted by Marc Eshel/Almaden/IBM] 
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Thomas Haynes <[email-masked]>
Date: Fri, 3 Oct 2014 15:41:21 -0400

> the other operations. I think we should have at least one implementation 

NetApp has a version of the copy offload on their Data ONTAP.

I think Andy has done a Linux version to test RPCSEC_GSSv3.

There is of course the LNFS work that is available with patches for Linux.

FWIW - waiting on implementation experience has been a major factor in the delay of shipping NFSv4.2.

---

From: Marc Eshel <[email-masked]>
Date: Fri, 3 Oct 2014 12:59:53 -0700

Thomas Haynes <[email-masked]> wrote on 10/03/2014 12:41:21 
PM:

> From: Thomas Haynes <[email-masked]>
> To: Marc Eshel/Almaden/IBM@IBMUS, 
> Cc: David Noveck <[email-masked]>, "[email-masked]" 
> <[email-masked]>, nfsv4 <[email-masked]>
> Date: 10/03/2014 12:42 PM
> Subject: Re: [nfsv4] proposed text for v4.2-based conversion to a 
> more incremental protocol extension model
> 

> I think Andy has done a Linux version to test RPCSEC_GSSv3.

Is there a client implementation?

> There is of course the LNFS work that is available with patches for 
Linux.

Yes, I knew but forgot about this one.

> in the delay of shipping NFSv4.2.

Anyone planes to implement WRITE_SAME ?

Marc.

---

From: Thomas Haynes <[email-masked]>
Date: Sat, 4 Oct 2014 00:17:07 -0400

> Is there a client implementation?

He needs a client and server to test.

> Anyone planes to implement WRITE_SAME ?

When this question has been asked before, two different companies have
indicated that they will.

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 4 Oct 2014 09:12:21 +0200

Two comments on this one:

 - how are operation and attribute number allocations coordinated?
 - Why not make this NFSv4.3 without actually any new features in the
   spec itself?  That gives us time to hash out any issues with it,
   and the above issue without blocking 4.2, which really should get
   out of the door ASAP.

---

From: Thomas Haynes <[email-masked]>
Date: Sat, 4 Oct 2014 11:12:43 -0400

>   and the above issue without blocking 4.2, which really should get

I'd actually like to see us pull the minorversioning out of 4.2 with a statement like:

At the time this document was specified, the minorversioning are detailed in
RFC5661. The implementer of the NFSv4.2 specification is expected to track
the evolution of the minorversioning standard.

I.e., steal what we did for avoiding specifying which flavor of Kerberos to use:

   At the time this document was specified, the Advanced Encryption
   Standard (AES) with HMAC-SHA1 was a REQUIRED algorithm set for
   Kerberos V5.  In contrast, when NFSv4.0 was first specified in
   [RFC3530], weaker algorithm sets were REQUIRED for Kerberos V5, and
   were REQUIRED in the NFSv4.0 specification, because the Kerberos V5
   specification at the time did not specify stronger algorithms.  The
   NFSv4 specification does not specify REQUIRED algorithms for Kerberos
   V5, and instead, the implementer is expected to track the evolution
   of the Kerberos V5 standard if and when stronger algorithms are
   specified.

I'd rather we not come out with a NFSv4.3 within a short period of NFSv4.2.

---

From: Christoph Hellwig <[email-masked]>
Date: Sat, 4 Oct 2014 17:53:44 +0200

> the evolution of the minorversioning standard.

That's fine with me, too - I just don't want NFSv4.2 blocked on sorting
out this modularization, which is going to require some though.  Most
importantly the first question in my previous mail.

---

From: David Noveck <[email-masked]>
Date: Sun, 5 Oct 2014 08:35:00 -0400

> - how are operation and attribute number allocations coordinated?

The idea is that the working group, will, at the time the I-D is to become
a working group document, look at the proposed assignments and make sure,
before agreeing to that status change, that there is no conflict with
existing working group documents for features in development.  I would
expect that those authoring feature I-D's would co-ordinate informally so
that the working group would rarely be presented with conflicts to resolve.

I thought about IANA but that is very high-overhead to do right and
pointless to do in half-hearted way.

---

From: David Noveck <[email-masked]>
Date: Sun, 5 Oct 2014 08:51:44 -0400

> I'd actually like to see us pull the minorversioning out of 4.2 with a
statement like:

Good idea.  Not only is it the proper response to the fact that minor
versioning rules
do not belong in individual minor version documents, it also makes the v4.2
spec shorter.

It puts me under the three-page limit.  How does minus-one pages sound? :-)

> the evolution of the minorversioning standard.

I think the problem may be that someone (for example, in the IESG) may ask
"what minor versioning standard?".    I expect we would have to convert
draft-haynes-nfsv4-minorversioning to a working group standards-track
document and have our co-chairs explain this to the IESG.   I hope we are
covered on charter-related issues.

---

From: Tom Haynes <[email-masked]>
Date: Sun, 5 Oct 2014 23:01:31 -0400

On Oct 5, 2014, at 8:35 AM, David Noveck <[email-masked]> wrote:

> The idea is that the working group, will, at the time the I-D is to become a working group document, look at the proposed assignments and make sure, before agreeing to that status change, that there is no conflict with existing working group documents for features in development.  I would expect that those authoring feature I-D's would co-ordinate informally so that the working group would rarely be presented with conflicts to resolve. 
> 
> I thought about IANA but that is very high-overhead to do right and pointless to do in half-hearted way.
> 
>  

But that is exactly what we do for new layout types and that does not seem too burdensome.

What I’d like to avoid is vendor A using operation X, never finalizing their specification, but getting a defacto assignment because code has shipped some customer might be using it.

I’d vote for IANA.

---

From: Benny Halevy <[email-masked]>
Date: Sun, 5 Oct 2014 23:43:53 -0400

The comparison to the pnfs layout type is justified. Yet, until ratified,
we defined a range for experimental layout typed to facilitate their
development and testing.

Similarly, we may want to officially define and reserve a space of
operation and attribute IDs for experimental use.

Benny
On Oct 5, 2014 8:01 PM, "Tom Haynes" <[email-masked]> wrote:

> What I’d like to avoid is vendor A using operation X, never finalizing
> their specification, but getting a defacto assignment because code has
> shipped some customer might be using it.
>
> I’d vote for IANA.
>

---

From: David Noveck <[email-masked]>
Date: Mon, 6 Oct 2014 09:29:38 -0400

Without enforcement powers, which we don't have and nobody is going to give
us, you cannot prevent irresponsible behavior.

Today, someone can create a new op, and ship clients and servers that agree
on that new op.  He is using the new op (de facto) but I don't think he has
a "de facto assignment".  He's on his own if he does that.

If you went the IANA route, he could still not bother to do the IANA
registration, or not bother with waiting for go through the (probably
required) expert review.  If he doesn't do those things he can use the new
op de facto, without a de jure assigment.

The same would happen if someone used a new op either didn't write a spec
or didn't wait for the working group to make it a working group document.
He would  use the new op de facto without a de jure assignment.

If you want to treat that situation as creating a de facto assignment you
can, but that's a choice you make and the same choice applies (about how
lax to be with those that ignore the rules) to all of these situations.

As I said, IANA is a high-overhead option.  It is very easy to write a
hand-wavy IANA request in the spec, but when people actually use it, the
problems arise.  You can't tell IANA to accept requests that make sense and
reject others.

---

From: David Noveck <[email-masked]>
Date: Mon, 6 Oct 2014 09:35:09 -0400

Yes you can, but you probably don't want an experimental range for each
flag bit and enumeration.

The trouble with experimental ranges is that you can create temporary
assigments for temporary use but as soon as more than one person is
involved, you have to have a way (preferably as simple as possible) to
create an asssigment that you can rely on.

On Sun, Oct 5, 2014 at 11:43 PM, Benny Halevy <[email-masked]>
wrote:

>> The idea is that the working group, will, at the time the I-D is to
>> become a working group document, look at the proposed assignments and make
>> sure, before agreeing to that status change, that there is no conflict with

---

From: Benny Halevy <[email-masked]>
Date: Mon, 6 Oct 2014 14:36:26 -0400

> flag bit and enumeration.

Not every flag, but save a mask for experimental usage to allow development
and testing of new features while in draft state.

> create an asssigment that you can rely on.

It's ok for the experimental implementation to collide as usually you don't
deploy and test them together, and you certainly don't want to encourage
that until they are finalized. Eventually, when they reach maturity we use
the RFC Editor annotations to mark all the places requiring working group
assignment of the final identifiers to be determined when the draft becomes
an RFC.  And RFC Editor is the gatekeeper asking the assignment from the WG
and enforcing that before approving the RFC.

Benny

---

From: Tom Haynes <[email-masked]>
Date: Mon, 6 Oct 2014 14:49:33 -0400

On Oct 5, 2014, at 8:51 AM, David Noveck <[email-masked]> wrote:

> Good idea.  Not only is it the proper response to the fact that minor versioning rules
> do not belong in individual minor version documents, it also makes the v4.2 spec shorter.
> 
> I think the problem may be that someone (for example, in the IESG) may ask "what minor versioning standard?".    I expect we would have to convert draft-haynes-nfsv4-minorversioning to a working group standards-track document and have our co-chairs explain this to the IESG.   I hope we are covered on charter-related issues.
> 

So, speaking of that draft, I’m going to push out an update with Dave’s micro versioning text.

---

From: Dave Quigley <[email-masked]>
Date: Wed, 08 Oct 2014 20:49:34 -0400

> Is there a client implementation?

So the answer is yes and no. When I was at the NSA I had contractors who 
implemented the then current RPCSEC_GSSv3 spec. I have the patches still 
and I have passed them along to several companies but I have yet to see 
anyone continue the work. The patches have long since bit rotted but I 
believe the implementation in them were sound.

Dave

> Yes, I knew but forgot about this one.

Also a bit of a clarification, Steve Dickson merged the patches we had 
for Labeled NFS into the main line kernel quite a few versions ago. We 
don't have RPCSEC_GSSv3 but we have simple attribute transport.

Dave

---

From: David Noveck <[email-masked]>
Date: Thu, 9 Oct 2014 08:30:18 -0400

>  I’m going to push out an update with Dave’s micro versioning text.

Just to clarify, I'm not proposing anything like 4.2.1.   A more
incremental versioning model would address the same issues that
micro-versioning was intended to, but it does so in a different way.

---

From: "Adamson, Andy" <[email-masked]>
Date: Mon, 13 Oct 2014 18:16:57 +0000

On Oct 8, 2014, at 8:49 PM, Dave Quigley <[email-masked]> wrote:

> So the answer is yes and no. When I was at the NSA I had contractors who implemented the then current RPCSEC_GSSv3 spec. I have the patches still and I have passed them along to several companies but I have yet to see anyone continue the work. The patches have long since bit rotted but I believe the implementation in them were sound.

The gssv3 spec has drastically changed - it is now a super set of gssv1 and gssv2. I have a prototype started….

—>Andy

> Also a bit of a clarification, Steve Dickson merged the patches we had for Labeled NFS into the main line kernel quite a few versions ago. We don't have RPCSEC_GSSv3 but we have simple attribute transport.
> 
> 
> Dave
>