Subject: [nfsv4] Additional WG chair and the WG future

From: Magnus Westerlund <[email-masked]>
Date: Fri, 25 Oct 2019 07:39:07 +0000

NFSv4 WG,

I am well aware of the frustration over the lack of progress in the WG. One of
the issues has been lack of WG chair activities in progressing and managing the
work items. As a first step to address that I want to recruit an additional WG
chair for the WG. Although that is not the only issue, it is one that will have
to be addressed directly.

Anyone interested in becoming a WG chair please contact me. I will not make a
decision within the next two weeks to allow people some time to seek support in
their organizations if they are considering taking on the role. I will solicit 
persons outside of the WG also, as I feel it is more important that we get
someone that has time to devote to run to processes, than necessary deep
technical knowledge of NFS. That the other chairs and the WG participants can
provide. 

Secondly, considering that the WG have a very small number of active
participants, and beyond these few there appears to be little energy. I am
deeply worried about the breadth of review this work gets. I think it is
important the WG focus on the most important work to finish up. And I seriously
think we need to consider if the right action is to close the WG, soon but in a
controlled fashion. Other WGs have done this in the past. The IETF has
procedures for maintenance and handling errata for existing specifications
beyond the WG active life. 

So this is a heads up to the WG, that several things will have to improve in the
WG if it is to go one any longer time. I know that in Montreal there discussion
of new work. But, as the situation stands I am extremely hesitant to add any new
milestones. Instead we need to show that the work currently taken on is dealt
with. If the WG wants to reprioritize existing work that can also improve the
situation. 

- Successful recruitment of a WG chair.
- Identification of what work is crucial to complete. 
- Good progress on the main items with both active editors and reviewers. 

If the above is shown in the next 6 months and there are still work the WG is
interested in then we can discuss adding new work. 

This is my view of the situation, and that is fairly dire. If WG participants
thinks the issues the WG faces can be addressed, I will listen to you. But
improvements need to be shown. 

Regards

Magnus Westerlund
TSV AD

---

From: David Noveck <[email-masked]>
Date: Fri, 25 Oct 2019 05:17:43 -0400

I understand why you are reluctant for the working group to take on new
work.  However, we need to make sure that existing work is properly dealt
with, including current items that do not have milestones.  Of particular
note in that regard is the completion of rfc5661bis (has no milestone but
the working group is pretty much committed to complete this as well as the
needed security enhancements to support it).  Given that there are a number
of errata (justifiably) REJECTED, assuming that the necessary changes would
be effected in rfc5661bis, I think the working group has to consider this
task as existing work notwithstanding the fact that there is not yet any
milestone for it.

I don't think we will be in a position to provide related milestones until
the IESG considerstion of rfc5661sesqui-msns is much farther along.

On Fri, Oct 25, 2019, 3:39 AM Magnus Westerlund <magnus.westerlund=
[email-masked]> wrote:

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 25 Oct 2019 10:20:08 +0000

Hi David, 

Yes, the fuller update to RFC 5661 I agree is likely one item, currently not a
milestone that is likely to be considered critical and thus it will be necessary
to add as a milestone. 

From my perspective, step one is to appoint an additional WG chair. At the same
time we will progress rfc5661sesqui-msns and see what response we get on it. 

At the same time the WG clearly have some WG items that are fairly close to be
ready to AD and IESG processing. Continue working on these dilligent the more
hopefully I will be that we can succeed in completing all the important item.
Which also makes it meaningful to have a discussion on how to deal with the
future. 

Cheers

Magnus

On Fri, 2019-10-25 at 05:17 -0400, David Noveck wrote:
> I understand why you are reluctant for the working group to take on new work. 
> However, we need to make sure that existing work is properly dealt with,
> including current items that do not have milestones.  Of particular note in
> that regard is the completion of rfc5661bis (has no milestone but the working
> group is pretty much committed to complete this as well as the needed security
> enhancements to support it).  Given that there are a number of errata
> (justifiably) REJECTED, assuming that the necessary changes would be effected
> in rfc5661bis, I think the working group has to consider this task as existing
> work notwithstanding the fact that there is not yet any milestone for it.
> 
> 
> I don't think we will be in a position to provide related milestones until the
> IESG considerstion of rfc5661sesqui-msns is much farther along.
> 
> On Fri, Oct 25, 2019, 3:39 AM Magnus Westerlund <
> magnus.westerlund=[email-masked]> wrote:
> > NFSv4 WG,
> > 
-- 
Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Networks, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Chuck Lever <[email-masked]>
Date: Fri, 25 Oct 2019 10:00:57 -0400

Hi Magnus-

I recognize that this is a difficult topic to bring up.

> On Oct 25, 2019, at 3:39 AM, Magnus Westerlund <magnus.westerlund=[email-masked]> wrote:
> 
> NFSv4 WG,
> 
> participants, and beyond these few there appears to be little energy.

However, it also needs to be said that the glacial WG process has been
one thing that has turned away potential WG participants I've spoken
with. Once the process issues are addressed, we might find that the WG
becomes more healthy and energetic.

> I am deeply worried about the breadth of review this work gets.

That's fair, but consider that this is a problem for any small WG.
Does the IESG have a plan for ensuring broader document review in
general?

> I think it is

Work that is crucial to complete:

Aside from the work on RFC 5661, there are existing milestones that
haven't been met or are unlikely to be met.

Date            Milestone 
Dec 2019	Submit final document defining RPC-over-RDMA Version 2 (as Proposed Standard)
Sep 2019	Submit final document describing pNFS RDMA Layout (as Proposed Standard)
Jun 2019	Submit final document describing CM private data convention for RPC-over-RDMA version 1 (Informational)

RPC-over-RDMA Version 2, as discussed in Montreal, has two
significant pieces of work remaining: support for host
authentication, and enhancements to its credit management mechanism.
Given the slow progress we've seen in WG process, it's not likely
it will be ready by December, even if we had closure on the
technical issues tomorrow.

The pNFS RDMA layout type work is highly desirable, but is currently
inactive, and its milestone data has already passed.

The CM private data convention milestone has also passed.

There is currently no milestone for RPC-on-TLS. This work has public
exposure, active authors, and active prototyping. In Montreal we
agreed to add a milestone for this, and I believe at this point we
have no choice but to add it.

There is no milestone for two current Working Group documents:
delstid and integrity-measurement. I think we should strive to
complete this work before a putative WG close down. IIRC there was
consensus in Montreal to add milestones for these. integrity-
measurement has an active author and is ready for WGLC, so there is
little WG effort needed to get it published.

Given the changes you're looking for and the amount of known work
there is left to do (rfc5661bis, and rpc-tls, at least) I think 6
months is aggressive... the bis, at least, is probably a multi-year
effort by itself.

NFS seems to have renewed relevance in cloud environments, thus
we certainly have been considering new work in this area. For
example:

- Even as we proposed RPC-on-TLS there have been complaints about
the known weaknesses in the TLS host authentication model.

- As QUIC matures and moves in to replace TCP, we will need to
respond with standards work to enable RPC on QUIC.

- We might want to consider supporting global authentication schemes
like OAuth to replace AUTH_SYS, which is still a widely-used yet
obsolete authentication scheme.

- NFS has very well-known performance weaknesses when it comes to
managing directory metadata. We have, in the past, considered
delegation to improve cache effectiveness, pNFS striping techniques
to increase throughput, and user space parallelism to help deal with
these weaknesses. It might be time to focus on protocol extensions
to help in this area.

A discussion of open technical issues and their stakeholders should
be part of the conversation about WG close down.

> improvements need to be shown.

IMO it is also important, as part of this conversation, to identify
the processes that will be used without a WG to extend the NFSv4
protocols and manage interoperability. The Linux NFS community, for
example, usually requires a standards document before considering
significant new features.

If we have new protocol work, where will it be proposed? Will we
need an IETF BoF to decide whether it is worth pursuing?

What will be the process for making authoritative changes to NFSv4?

Can you post some BCPs or other documentation that describes what
this might look like?

--
Chuck Lever

---

From: David Noveck <[email-masked]>
Date: Sat, 26 Oct 2019 12:17:30 -0400

> becomes more healthy and energetic.

That certainly is an important issue, and I feel that addition of a co-chair
will address it.   However, there are other things that should be done to
improve the situation.   Of particular concern to me is situation with
working
group meetings.   The general assumption is that we will not be meeting and
that
a decision to meet is only to be taken after creating a draft meeting
agenda.
While this happens early enough that a few people deeply involved in the
working
group can secure financial approval to make the trip, it definitely limits
the set
of people who can attend who are not within driving distance of the meeting
site.
It is easier to make the case for attendence if one knows months in advance
and one
can co-ordinate constructively with other activities in the general area.

I also think we need to do a better job of engaging more people involved in
implementing
nfsv4.   I know there are process difficulties but I think we should be
able to conduct
non-sub rosa working-group-related discussion at NFSv4 testing events.   In
addition,
although there has been dicussion of having conference calls to supplement
the
working group meetings at IETF meetings, they rarely materialize.   We
should not be
so hesitant to go beyond mailing list discussions, and should also consider
interim meetings
near were many participants are located, supplemented by remote
participation.   In my
experience the discussion on the working group list is helpful but it is
hard to move to a decision.
An interim meeting is an occasion for someone to say "OK, but when is that
going to be done?"
and I think we need that sometimes to move forward more expeditiously.

> That's fair, but consider that this is a problem for any small WG.

I think we have to make it our business to get appropriate review.   The
person
primarily responsibe for getting appropriate review before WGLC should be
the
document editor but there is a need for the chair to act as a backstop (no
connection
with Brexit :-)

> > I think it is
> > important the WG focus on the most important work to finish up.

Agree, but note that one subset set of the important work (rfc65661bis and
things
closely related to it) is quite large.

>> And I seriously
>> controlled fashion.

It depends on what you mean by "soon".   If we need to complete rfc5661bis
and
I believe we do then any controlled shutdown would take quite a while.

However, having considered the matter, as Magnus suggested we should, I
hope it
doesn't come to that.

>> Other WGs have done this in the past. The IETF has

I'm sure it does, but I'm not sure how effective they would be in
addressing the
needs of the NFSv4 community.

> > WG if it is to go one any longer time.

Fair enough.

>> I know that in Montreal there discussion
>> of new work.

Yes, but the situstion was confusing in that some of that new work is for
items that currently have milestones and other pieces were part of tasks,
such as completion of rfc5661bis that have no milestones (yet) but are
generally recognized as being necessary.

> > But, as the situation stands I am extremely hesitant to add any new
> > milestones.

Understood.

>> Instead we need to show that the work currently taken on is dealt
>> with.

That makes sense to me.

>> If the WG wants to reprioritize existing work that can also improve the
>> situation.

Believe there are some opportunities.  See below.

> > - Successful recruitment of a WG chair.

Clearly essential.

> > - Identification of what work is crucial to complete.

I think we need to discuss this on the working group list.   If
that discussion does not conclude with an agreed plan in the
next 4-6 weeks, will need to address this in a conference call
and come up with a working plan.

>> - Good progress on the main items with both active editors and reviewers.

I agree we need to focus on this.

>> interested in then we can discuss adding new work.

I think we should be quite convervative in that regard since, even with
good progress,
we will not be all that close to finishing all the important items.

> Jun 2019        Submit final document describing CM private data
convention for RPC-over-RDMA version 1
>(Informational)

> significant pieces of work remaining:
>      support for host authentication,

That is clearly security-related and should be considered essential.

>      enhancements to its credit management mechanism.

That should only be considered essential if is necessary to effect host
authentication.

We should consider the possibility of a smaller v2, if that is necessary
and possible with the expectation that we would address send chaining
(highly desirable) as new work later.

> technical issues tomorrow.

Agree that the December date is unrealizable.   We should try to come up
with a realistic date (assuming the shepherding issue is properly
addressed) and let that drive the decision on the scope of v2.

> inactive, and its milestone data has already passed.

I think it makes sense to simply drop this from the milestone list.

With regard to the NVMe layout work, I think we need to decide whether the
existing milestone, 4/2020, is realistic.   I generally feel that six
months after
a submission of a working group document is already fairly aggressive.   In
this case, there isn't even an I-D and have no idea when one will be
available.
We should set a new milestone date for this when the I-D is submitted.

> The CM private data convention milestone has also passed.

You finished this on 6/13 so it could have met its milestone, if the
working group shepharding function was working as it should.  As things
have turned out it's had three different shepherds, the last of whom, Beepee
started on 10/2.

I think we should ask Brian to set a new, realistic milestone date.   If he
is
unable to do so, this will wind up on the new chair's plate.   In no case,
can I see
letting this drag on past the end of the year.

> have no choice but to add it.

So, when do you and Trond anticipate coming up with the milestone?   It
would
be good if we could incorporate it, with a milestone, in the list of
essential items
to be worked on.

> complete this work before a putative WG close down.

I think I need to do a review of delstid.  If this is to be considered
essential, it would help to have a realistic milestone for it.

> IIRC there was
> consensus in Montreal to add milestones for these.

Whether there was or not, there is no possibility of including
things on the list of essential items with neither a target date
not a clear expectation when one will be available.

> integrity-

BTW, I owe you a complete review of this document.  Expect it in
the next two weeks.

> months is aggressive...

I think six months is time for us to show Magnus that we can get our
house in order.   I don't think Magnus is asking us to complete all this
work
in six months

> the bis, at least, is probably a multi-year
> effort by itself.

Yes.

>> This is my view of the situation, and that is fairly dire.

True.   The biggest problem is that we have come to view the
current state of affairs as normal/expected.   I think you've made it
clear that that has to change.

> If WG participants
> thinks the issues the WG faces can be addressed, I will listen to you.

I expect so.

>> But

Fair enough.

> protocols and manage interoperability.

That's a separate conversation and I don't see much point pursuing
it on the framework of the NFSv4 working group.   If the working group
does go away, we'll do the best we can, then.

> The Linux NFS community, for

As our experience with NFSv4.2 has shown, that is not sufficient.  Given
the importance of the Linux client's role, any issues that implementors have
with the spec need to be identified earlier, rather than years after
publication.

> If we have new protocol work, where will it be proposed?

Almost anywhere.   I anticipate the extension model dissolving into chaos
:-(

> Will we

Nobody would bother to attend such a BoF anyway.

I think we all need to work together to avoid this sort of nightmare.

> What will be the process for making authoritative changes to NFSv4?

We don't know.

> this might look like.

Such documents are highly unlikely to exist.  Normally working groups
expire when they have nothing to do.  The case of a working group ceasing
to exist because the necessary work was not adequately attended to is
probably one nobody in the IETF really wants to think about.

On Fri, Oct 25, 2019 at 10:01 AM Chuck Lever <[email-masked]> wrote:

> Jun 2019        Submit final document describing CM private data
> convention for RPC-over-RDMA version 1 (Informational)
>