Subject: Re: [nfsv4] rfc5666bis and rpcrdma-bidirection progress update

From: William Allen Simpson <[email-masked]>
Date: Mon, 21 Nov 2016 15:32:30 -0500

Just got the notice that the draft is about to expire next week.

Assuming that we need a new draft, so as to not interrupt the
excruciatingly sluggish review process.

---

From: Chuck Lever <[email-masked]>
Date: Mon, 21 Nov 2016 15:45:15 -0500

Bill-

I also got this notice. I'll take care of it before Wednesday.

> On Nov 21, 2016, at 3:32 PM, William Allen Simpson <[email-masked]> wrote:
> 
> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4

--
Chuck Lever

---

From: David Noveck <[email-masked]>
Date: Sun, 27 Nov 2016 06:55:29 -0500

So the review process will not be interrupted.  It will continue at the
same pace as before.

With regard to the characterization of the process as "excruciatingly
sluggish", it does seem that not very much has happened since the working
group reached consensus on  this document on 4/17/2016 (according to the
data tracker history).

However, to be fair, we should note that draft-ietf-nfsv4-rfc5666bis-00 was
submitted on 12/01/2015 and it it took the working group less than a year
to get to the point we are.    It looks likely to me that the whole process
will (cross your fingers) be completed in around 16 months from first
submission.   It is hard to get documents done faster than that within the
IETF process, which is not optimized for speed.

Contrast that with RFC5666 itself which took over five years from -00
submission to RFC publication.  Despite the slow pace of the process, it
appears that the document did not get the kind of review it needed before
publication, making rfc5666bis necessary. I'd like to thank the authors for
persevering in the effort to provide NFS with an RDMA-based transport.

If we want to improve the process, and I believe we should, we need to
focus on review quality/adequacy as well as the pace of the process.

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 28 Nov 2016 16:53:44 +0000

For this comment, “If we want to improve the process, and I believe we should, we need to focus on review quality/adequacy as well as the pace of the process.”

If there is a desire for an item to move more quickly, treat it like any other item in life – take responsibility for it, invest the time, get others that are needed to do the same.

There are a variety of techniques for managing work that is done by others.  Reminders, regularly scheduled conference calls, changing ownership, breaking the tasks into smaller items, discarding unneeded functionality to move the process forward.

The “process” can move more quickly if desired – at least for the items that are under direct control and in the case of Internet Drafts – most of it is under the control of the authors and the collective working group once there is consensus that an I-D is a WG draft.

Spencer

From: nfsv4 [mailto:[email-masked]] On Behalf Of David Noveck
Sent: Sunday, November 27, 2016 3:55 AM
To: Chuck Lever <[email-masked]>
Cc: [email-masked]
Subject: Re: [nfsv4] rfc5666bis and rpcrdma-bidirection progress update

So the review process will not be interrupted.  It will continue at the same pace as before.

With regard to the characterization of the process as "excruciatingly sluggish", it does seem that not very much has happened since the working group reached consensus on  this document on 4/17/2016 (according to the data tracker history).

However, to be fair, we should note that draft-ietf-nfsv4-rfc5666bis-00 was submitted on 12/01/2015 and it it took the working group less than a year to get to the point we are.    It looks likely to me that the whole process will (cross your fingers) be completed in around 16 months from first submission.   It is hard to get documents done faster than that within the IETF process, which is not optimized for speed.

Contrast that with RFC5666 itself which took over five years from -00 submission to RFC publication.  Despite the slow pace of the process, it appears that the document did not get the kind of review it needed before publication, making rfc5666bis necessary. I'd like to thank the authors for persevering in the effort to provide NFS with an RDMA-based transport.

If we want to improve the process, and I believe we should, we need to focus on review quality/adequacy as well as the pace of the process.

---

From: David Noveck <[email-masked]>
Date: Mon, 28 Nov 2016 12:32:58 -0500

With regard to these documents I suppose that it is appropriate to remind
you of the history here:

   - These documents have been in the state WG Consensus:Waiting for
   Writeup for over four months, since 7/19.
   - Chuck tried to reduce the workload by deciding to drop the associated
   informational document.
   - Chuck made efforts to remind you of the need to provide these writeups.
   - This appeared to culminate in your statement  on 10/25: " For the I-Ds
   that are "waiting for write-up", those are on my to-do list and will be out
   in the next week"
   - It is now about five weeks since that statement and we have heard
   nothing about progress in this regard.

> Reminders, regularly scheduled conference calls, changing ownership,
breaking the tasks into smaller items, discarding unneeded functionality to
move the process forward.

We've tried reminders and Chuck did remove unnecessary work.  However, the
IETF rules don't appear to allow changing ownership of the task of
producing a wrIteup.  If you think that is appropriate, then perhaps you
should investigate it.  Given that there is a bunch of other documents that
will soon be in the same state, maybe you should consider that.  Would
regularly scheduled conference help, or would they just take up time?

> most of it is under the control of the authors and the collective working
group once there is consensus that an I-D is a WG draft.

True, but there is a significant portion of the process where we wind up
waiting for action by the WG chair, and for these documents, that has been
a considerable portion of the delay.  Is there a viable way to spread out
this work better?

On Mon, Nov 28, 2016 at 11:53 AM, Spencer Shepler <[email-masked]>
wrote:

>
>
> For this comment, “If we want to improve the process, and I believe we
> should, we need to focus on review quality/adequacy as well as the pace of
> the process.”
>
>
>
> If there is a desire for an item to move more quickly, treat it like any
> other item in life – take responsibility for it, invest the time, get
> others that are needed to do the same.
>
>
>
> There are a variety of techniques for managing work that is done by
> others.  Reminders, regularly scheduled conference calls, changing
> ownership, breaking the tasks into smaller items, discarding unneeded
> functionality to move the process forward.
>
>
>
> The “process” can move more quickly if desired – at least for the items
> that are under direct control and in the case of Internet Drafts – most of
> it is under the control of the authors and the collective working group
> once there is consensus that an I-D is a WG draft.
>
>
>
> Spencer
>
>
>
> *From:* nfsv4 [mailto:[email-masked]] *On Behalf Of *David Noveck
> *Sent:* Sunday, November 27, 2016 3:55 AM
> *To:* Chuck Lever <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* Re: [nfsv4] rfc5666bis and rpcrdma-bidirection progress update
>
>
>
> However, to be fair, we should note that draft-ietf-nfsv4-rfc5666bis-00
> was submitted on 12/01/2015 and it it took the working group less than a
> year to get to the point we are.    It looks likely to me that the whole
> process will (cross your fingers) be completed in around 16 months from
> first submission.   It is hard to get documents done faster than that
> within the IETF process, which is not optimized for speed.
>

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 28 Nov 2016 18:12:27 +0000

My comments were general, Dave.  Yes, I am the bottleneck on some of this but I get one nag email every few weeks.  Mmm, yes, it has slipped on my todo list but I also have not been reminded with any great frequency – is it urgent or not.

Yes, I can improve my cycle time but there are many other steps involved and just waving that it is “IETF Process” is unsatisfactory from my point of view.  It is what you make of it. The elapsed time can be reduced – unfortunately, much of the slowness is self-induced (working group self-induced) and indicative of the relative priorities of everyone involved.

Spencer

From: David Noveck [mailto:[email-masked]]
Sent: Monday, November 28, 2016 9:33 AM
To: Spencer Shepler <[email-masked]>
Cc: Chuck Lever <[email-masked]>; [email-masked]
Subject: Re: [nfsv4] rfc5666bis and rpcrdma-bidirection progress update

With regard to these documents I suppose that it is appropriate to remind you of the history here:

  *   These documents have been in the state WG Consensus:Waiting for Writeup for over four months, since 7/19.
  *   Chuck tried to reduce the workload by deciding to drop the associated informational document.
  *   Chuck made efforts to remind you of the need to provide these writeups.
  *   This appeared to culminate in your statement  on 10/25: " For the I-Ds that are "waiting for write-up", those are on my to-do list and will be out in the next week"
  *   It is now about five weeks since that statement and we have heard nothing about progress in this regard.
> Reminders, regularly scheduled conference calls, changing ownership, breaking the tasks into smaller items, discarding unneeded functionality to move the process forward.

We've tried reminders and Chuck did remove unnecessary work.  However, the IETF rules don't appear to allow changing ownership of the task of producing a wrIteup.  If you think that is appropriate, then perhaps you should investigate it.  Given that there is a bunch of other documents that will soon be in the same state, maybe you should consider that.  Would regularly scheduled conference help, or would they just take up time?

> most of it is under the control of the authors and the collective working group once there is consensus that an I-D is a WG draft.

True, but there is a significant portion of the process where we wind up waiting for action by the WG chair, and for these documents, that has been a considerable portion of the delay.  Is there a viable way to spread out this work better?

On Mon, Nov 28, 2016 at 11:53 AM, Spencer Shepler <[email-masked]<mailto:[email-masked]>> wrote:

Spencer

From: nfsv4 [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of David Noveck
Sent: Sunday, November 27, 2016 3:55 AM
To: Chuck Lever <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [nfsv4] rfc5666bis and rpcrdma-bidirection progress update

---

From: David Noveck <[email-masked]>
Date: Mon, 28 Nov 2016 13:47:40 -0500

> Yes, I am the bottleneck on some of this but I get one nag email every
few weeks.

It appears that you will soon be getting more, as the implication of your
remarks is that, if you are not reminded frequently, things will be judged
non-urgent and simply not get done.

> Mmm, yes, it has slipped on my todo list but I also have not been
reminded with any great frequency

What frequency of reminder would be necessary to get to do what you have
already committed yourself to do?

I can remind you weekly or daily, if you choose.

> is it urgent or not.

Well, nobody's life depends on it.  But, we have a serious problem with
regard to our RDMA transport and Chuck and others have devoted considerable
work to rectifying it.   You are, unintentionally, standing in the way of
that work coming to fruition.  As understand it, that is why Chuck reminded
you of the need to proceed on this and why you told him it would get done
soon.  I don't think  that Chuck needs to take on the work of reminding you
to do what you have said you will do.   I think it would make more sense if
you, when you became unable to do what you said you would, asked Chuck
about priorities.

> Yes, I can improve my cycle time but there are many other steps involved
and just waving that it is “IETF Process” is unsatisfactory from my point
of view.

I've tried to break it up to specific elements, and I may have appeared
unfair in pointing to areas where you were the bottleneck.

There has been difficulty in getting people to actually read and comment on
some of the extension documents, but, beyond pointing out that fact, I
don't see what else I can do.

> It is what you make of it. The elapsed time can be reduced –
unfortunately, much of the slowness is self-induced (working group
self-induced)

I suppose you mean "induced by others than the WG chair", in which case I
agree.

> indicative of the relative priorities of everyone involved.

True but in the particular case of the RDMA documents, the authors and the
working group have treated this as  of high priority while you have not,
since you were not pelted with frequent reminders. I think that is the
source of complaints about the "excrucuiating sluggishness" of the process.

For other documents, the balance has been different.

On Mon, Nov 28, 2016 at 1:12 PM, Spencer Shepler <[email-masked]>
wrote:

>
>
> My comments were general, Dave.  Yes, I am the bottleneck on some of this
> but I get one nag email every few weeks.  Mmm, yes, it has slipped on my
> todo list but I also have not been reminded with any great frequency – is
> it urgent or not.
>
>
>
> Yes, I can improve my cycle time but there are many other steps involved
> and just waving that it is “IETF Process” is unsatisfactory from my point
> of view.  It is what you make of it. The elapsed time can be reduced –
> unfortunately, much of the slowness is self-induced (working group
> self-induced) and indicative of the relative priorities of everyone
> involved.
>
>
>
> Spencer
>
>
>
> *From:* David Noveck [mailto:[email-masked]]
> *Sent:* Monday, November 28, 2016 9:33 AM
> *To:* Spencer Shepler <[email-masked]>
> *Cc:* Chuck Lever <[email-masked]>; [email-masked]
>
>    - Chuck tried to reduce the workload by deciding to drop the
>    associated informational document.
>    - This appeared to culminate in your statement  on 10/25: " For the
>    I-Ds that are "waiting for write-up", those are on my to-do list and will
>    be out in the next week"
> > most of it is under the control of the authors and the collective
> working group once there is consensus that an I-D is a WG draft.
>
>
>

---

From: Chuck Lever <[email-masked]>
Date: Mon, 28 Nov 2016 14:32:00 -0500

> On Nov 28, 2016, at 1:12 PM, Spencer Shepler <[email-masked]> wrote:
> 
>  

I was informed several months ago that there were at least five I-Ds in the
WG pipeline ahead of rfc5666bis and rpcrdma-bidirection. Three of them have
recently become RFCs, leaving at least two more. The remaining documents I'm
aware of are the multi-domain I-D and the SCSI layout I-D.

So far I haven't felt it necessary to send frequent reminders about my I-Ds
because chair write-ups are not really what's blocking the progress of these
documents. I didn't feel it necessary to nag someone who couldn't do much
about the delays with RPCSEC GSSv3, for example.

It did leave me with a sense of helplessness about this backlog. And I can
admit to having spent less time recently on IETF work because it didn't
seem valuable to pursue, given the existing blockages. Lack of nagging was
not related to relative priorities.

It would help to get regular status updates on the document pipeline,
whether there are any hold-ups, and what the non-executive members of the
WG can do about them. IESG review processes are managed off the WG mailing
list, thus many of us are not privy to progress on individual I-Ds, nor do
we have a sense of what tasks there are remaining to do. The chairs are
usually copied on all of this. Perhaps they could monitor and periodically
reflect a summary to the WG.

> 	• These documents have been in the state WG Consensus:Waiting for Writeup for over four months, since 7/19.
> 	• Chuck tried to reduce the workload by deciding to drop the associated informational document.
> 	• Chuck made efforts to remind you of the need to provide these writeups.
> 	• This appeared to culminate in your statement  on 10/25: " For the I-Ds that are "waiting for write-up", those are on my to-do list and will be out in the next week"
> 	• It is now about five weeks since that statement and we have heard nothing about progress in this regard.  
> On Mon, Nov 28, 2016 at 11:53 AM, Spencer Shepler <[email-masked]> wrote:
>  

--
Chuck Lever

---

From: Spencer Shepler <[email-masked]>
Date: Mon, 28 Nov 2016 22:33:51 +0000

Chuck,

You comment:
"It would help to get regular status updates on the document pipeline, whether there are any hold-ups, and what the non-executive members of the WG can do about them. IESG review processes are managed off the WG mailing list, thus many of us are not privy to progress on individual I-Ds, nor do we have a sense of what tasks there are remaining to do. The chairs are usually copied on all of this. Perhaps they could monitor and periodically reflect a summary to the WG.[Spencer] "

Sure, I can work to push out a summary for those time where documents are hung up on procedural or review comments, etc.

For those that are interested to follow along, all document status for the WG can be found here:

https://datatracker.ietf.org/group/nfsv4/documents/

Current state of the document, next steps, and what they mean are available via these pages.

For an individual document that has been or is going through IESG the specific comments and feedback are captured.  For example, the iSCSI layout document comments are listed here: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-scsi-layout/ballot/

Spencer

---

From: William Allen Simpson <[email-masked]>
Date: Tue, 29 Nov 2016 02:23:21 -0500

Why is this message in Spam? It has a from address in microsoft.com but
has failed microsoft.com's required tests for authentication.

(For all your messages today.  Thank goodness I check spam every so often.)

---

From: David Noveck <[email-masked]>
Date: Tue, 29 Nov 2016 04:50:57 -0500

It is possible to get around this in gmail by creating a filter. One of the
options, for a defined set of messages, such as messages from a given
address, is to not treat the message as spam. Then it appears in your inbox
normally, except it is preceded by a little note: Why is this message not
in spam? :-)

On Nov 29, 2016 2:23 AM, "William Allen Simpson" <
[email-masked]> wrote:

>> "It would help to get regular status updates on the document pipeline,
>> reflect a summary to the WG.[Spencer] "
>>
>> Sure, I can work to push out a summary for those time where documents are
>> hung up on procedural or review comments, etc.
>>
>> For an individual document that has been or is going through IESG the
>> specific comments and feedback are captured.  For example, the iSCSI layout
>> document comments are listed here: https://datatracker.ietf.org/d
>> oc/draft-ietf-nfsv4-scsi-layout/ballot/
>>
>> Spencer
>>
>>
>>

---

From: Chuck Lever <[email-masked]>
Date: Tue, 29 Nov 2016 10:40:14 -0500

> On Nov 28, 2016, at 5:33 PM, Spencer Shepler <[email-masked]> wrote:
> 
> 
> Chuck,
> 
> You comment:

Much appreciated, Spencer!

--
Chuck Lever

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 29 Nov 2016 12:52:23 -0600

> should investigate it.  Given that there is a bunch of other documents that

The "IETF rules" (insamuch as there are rules) do allow for changing the
document shepherd after assignment, and for non-WG-chair document shepherds.
The chairs have a fair amount of latitude in assigning shepherds, as I understand it.
We've been having similar problems with a backlog of documents in the kitten
WG, and recently had a contributor volunteer some time to help shepherd
documents, which has made a big difference.

> a considerable portion of the delay.  Is there a viable way to spread out

Bringing in non-chair document shepherds might help, and there is also
some possibility (in terms process) of adding a third co-chair, if there
are suitable candidates and the current chairs/AD are amenable.

Ultimately, it comes down to finding someone willing to do the work who
has time; shepherding is some of the least glorious work (you don't even
get your name on the final RFC!), but is still an important part of
the document lifecycle.

I see that the two documents in the subject have already been submitted
for publication, so maybe there is no longer an immediate need, but it
still seemed worth pointint out that there are a few options in this space.

-Ben

---

From: David Noveck <[email-masked]>
Date: Tue, 29 Nov 2016 14:35:40 -0500

Perhaps we can revisit your suggestions when some of the documents
currently in WGLC (three!) require shepherding.  For a very long time,
Spencer has been the only person doing shepherding and I don't feel
that this situation can go on forever.

---

From: William Allen Simpson <[email-masked]>
Date: Wed, 30 Nov 2016 06:23:43 -0500

> that this situation can go on forever.
Speaking as the NSFnet oversight person who initiated the overthrow of
the IAB because of their inability to approve RFCs and WG creation in a
timely manner (6 months).

And as a member of both the POISED and POISSON WGs that wrote the
IETF process.

There is a reason that 6 months is written into our standards process --
that is the minimal time for complete delivery of an RFC changing the
status of a draft, to Proposed and then to (now) Full Standard.  It's
enough time for review.  We said so fairly clearly.

There is a reason that we picked 24 months for failure to mature.

When it comes to Last Call, Spencer is just a member of the WG.  His
comments are due at the same time as everybody else.

Shepharding is just prompting the IESG to get the items on their
agenda.  It shouldn't take more than one person....

---

From: David Noveck <[email-masked]>
Date: Wed, 30 Nov 2016 10:09:30 -0500

> enough time for review.  We said so fairly clearly.

Certainly I've never seen a document published in less than this minimal
time.

Most documents take considerably more.  In some cases I'm not clear about
the reasons.

> There is a reason that we picked 24 months for failure to mature.

I'm not sure of that reason or what the implications of "failure to mature"
might be.

I've seen a lot of documents take longer than this.  The record is mixed:

   - RFC7530 took 71 months to go from -00 to publication.  It matured
   slowly but it did mature.
   - RFC5661 took 51 months to go from -00 to publication.  Given the size
   of the document, and the scope of the changes, I think that time is
   unexceptionable.
   - RFC5666 took 66 months to go from -00 to publication.  As we have
   seen, there are some problems with this document and it appears that it not
   get adequate review but I don't see that stopping things arbitrarily after
   24 months (in July 2006) would have made things better.  When people are
   rushing to meet a deadline, it tends to hurt the quality of review rather
   than helping it.
   - RFC7862 took 67 months to go from -00 to publication.  I feel that
   this length of time was the result of jamming together too many unrelated
   extensions, and the working group is now in the process of trying to
   address this issue going forward
   - draft-ietf-nfsv4-versioning will have taken 25 months to go from -00
   to the end of WGLC. It took a while to mature and I would have liked it to
   happen faster but I believe ithis document  should go forward promptly once
   WGLC is complete.

> When it comes to Last Call, Spencer is just a member of the WG.

Yes but we have often waited for him to start it.  Is that correct?  Should
authors, in consultation with rest of the working group, simply
schedule WGLC on their own.? Of course it may not be possible for others to
change the document state.

There are limits to the IETF's commitment to egalitarian ideals.  As Boss
Tweed said, "I don't care how the people vote, as long as I get to do the
nominating."

> His comments are due at the same time as everybody else.

True.  That implies that if he has issues with the document, he should send
them to the list.

On the other hand, RFC 4858 requires the shepherd to state, separately,
that he has read the document and does not know of such issues.  So it
appear that this sensible view is not shared universally.

> Shepherding is just prompting the IESG to get the items on their agenda.
It shouldn't take more than one person....

If you read RFC 4858 you get a different view.  Of course that document
says it is offered on an "AS-IS" basis, and might not be fit for any
particular purpose.

Some people assume shepherding is more extensive than you do.  Spencer
seemed to imply it was extensive but, once it became clear that this was
important, the needed documents were produced quite quickly.

My advice to authors is to to give a document a week waiting for a
write-up, and then proceed to extensive nag mail, as Spencer takes the lack
of such mail as indicating the write-up can be put off indefinitely.

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 30 Nov 2016 16:22:41 -0600

> agenda.  It shouldn't take more than one person....

I must disagree that shepherding is "just prompting the IESG".  See, for example,
https://www.ietf.org/iesg/template/doc-writeup-qa-style.html, which includes
several questions that should be answered by the shepherd (if relevant for the
document at hand).  In particular, the shepherd should provide a summary of
the WG review process, whether there was controversy about various points, etc.;
when there are many documents in the WG and the WG review was spread out over
time, this can involve reviewing the entire mailing list archive of the document's
discussions, which does take a noticable amount of time.
It also implies that the shepherd should perform validation of the sections
of the document written in formal language such as XDR, BNF, etc..

-Ben