Subject: Proposal for a significant change to the shepherd writeup

From: Barry Leiba <[email-masked]>
Date: Thu, 20 Sep 2012 14:05:39 -0400

Chairs...

The IESG has been talking about a change to the document shepherd
writeup, to take it away from a Q/A format, and more into a format
that gives information and focuses on exceptions, controversies, and
other things the IESG specifically needs to know about.  A proposal
for that is on the IESG wiki here:

   http://tools.ietf.org/group/iesg/trac/wiki/DraftShepherdWriteupWgAlternate

The idea, as is explained there, is that there's a checklist that the
shepherd should go through, which won't itself be submitted.  The
expected answer to every question in the checklist is "yes", and if
any answer is not an unequivocal "yes", that should lead to some
explanation in the writeup itself.

The writeup, instead of being answers to a bunch of questions, some of
which were strange or confusing or repetitious or repetitious or
repe..., would be some explanatory text in four sections.  Now,
instead of having to say "No" to "Does the shepherd have any
particular concerns about the document?", or "Yes" to "Has the
shepherd checked this or that?", nothing at all need be said.  If
there was controversy about something that divided the working group
and was difficult to resolve, an explanation of that would go into
Section 2 of the writeup.

There are a couple of examples linked at the top of the page, showing
how the writeup might come out with this proposal.  The goal is to
make sure the writeup contains the information the IESG needs, while
actually making it easier to fill out in some ways (though that will
clearly depend upon how much detail is going into the writeups now).

We do not currently want chairs/shepherds to use this format in
general.  We will be asking for trial use of this, as an experiment,
for selected documents, and you'll get specific requests for that.
Otherwise, we're interested in the chairs' feedback, as you have a
look at the proposal.  You can post your feedback here, on the
wgchairs list.

Thanks,
Barry, and the IESG

---

From: Bob Hinden <[email-masked]>
Date: Thu, 20 Sep 2012 15:20:43 -0700

Barry,

At first glance, I liked this proposal, but as I read through it I think it's going to be a lot more work for the person writing it up.  If I am going to have to go through the checklist, it would be just as easy to keep in in the current form.  I suspect everyone does what I do, that is, they copy it into their editor and answer the questions.  It's very straight forward.  One does not have to check if anything is missing, etc.

My 2 cents.

Bob

---

From: <[email-masked]>
Date: Thu, 20 Sep 2012 22:27:44 +0000

+1 on Bob's remarks.

Does the IESG feel that the current questionnaire/proto writeup
insufficient?

Some of the questions in the current writeup are not applicable to all WGs
or documents, but it is not a problem to respond with a N/A from the
chairs perspective.

-Raj

On 9/20/12 5:20 PM, "ext Bob Hinden" <[email-masked]> wrote:

>Barry,
>
>At first glance, I liked this proposal, but as I read through it I think
>it's going to be a lot more work for the person writing it up.  If I am
>going to have to go through the checklist, it would be just as easy to
>keep in in the current form.  I suspect everyone does what I do, that is,
>they copy it into their editor and answer the questions.  It's very
>straight forward.  One does not have to check if anything is missing, etc.
>
>My 2 cents.
>
>Bob
>

---

From: Acee Lindem <[email-masked]>
Date: Thu, 20 Sep 2012 19:13:04 -0400

+1 - Also, I believe the shepherding template was recently updated (within the last 1-2 years). 
Thanks,
Acee

---

From: John C Klensin <[email-masked]>
Date: Thu, 20 Sep 2012 19:27:36 -0400

> updated (within the last 1-2 years). 

And, IMO, got worse (more redundant and more questions that
shouldn't have needed to be asked).

FWIW, I've tried out the new writeup format (on one of those
"selected documents") and have distinctly mixed feelings about
it even though I'm completely sympathetic with the intend.  I'll
hold off on more detailed comments until others have had a
chance to look at it and comment.

   john

---

From: Mary Barnes <[email-masked]>
Date: Thu, 20 Sep 2012 18:31:21 -0500

I totally agree.  I didn't see information in the new proposal that
couldn't easily be included in the current write-up and of course, "yes",
"no" and "N/A" are valid responses, so I don't think the current template
is a problem.

Mary.

On Thu, Sep 20, 2012 at 5:20 PM, Bob Hinden <[email-masked]> wrote:

> going to have to go through the checklist, it would be just as easy to keep
> in in the current form.  I suspect everyone does what I do, that is, they
> copy it into their editor and answer the questions.  It's very straight
> forward.  One does not have to check if anything is missing, etc.
>
> My 2 cents.
>
> Bob
>

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Fri, 21 Sep 2012 11:36:03 +0200

I agree with Bob and Mary. 

Maybe it's just me being used with the questionnaire format and having used it for many years both as a shepherd and as an AD. I prefer as a shepherd explicitly going through the list and I do not believe that writing a 'Yes', 'No' or 'N/a' represents any significant overhead. As an AD I liked the fact that I could find the same information at the same place always, and that I got the positive confirmation that the shepherds scanned the full checklist. 

Seriously, what problems are we trying to solve? Barry wrote: 

Ø  .  The goal is to
> clearly depend upon how much detail is going into the writeups now).

I do not believe that the new mode of writing is easier to fill out. The accuracy and level of details still depends on the shepherds. A good shepherd will fill in all relevant information on the critical technical and process issues whatever form is used, is the concern that this information risks to be lost in the existing form? 

The last change in the form dates less than seven months, the current version at http://www.ietf.org/iesg/template/doc-writeup.txt is dated 2/24/12.

I am also missing from Barry's proposal the Document Announcement write-up. Does he suggest this should not be any longer part of the write-up?  Will the ADs now be responsible on creating those? 

Regards,

Dan

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mary Barnes
Sent: Friday, September 21, 2012 2:31 AM
To: Bob Hinden
Cc: [email-masked]; Barry Leiba
Subject: Re: Proposal for a significant change to the shepherd writeup

I totally agree.  I didn't see information in the new proposal that couldn't easily be included in the current write-up and of course, "yes", "no" and "N/A" are valid responses, so I don't think the current template is a problem. 

Mary. 

Barry,

My 2 cents.

Bob

---

From: Jari Arkko <[email-masked]>
Date: Tue, 25 Sep 2012 21:07:47 +0300

I'm with Bob, too.

But I have a slightly different, additional, angle. When I saw the e-mail, and the rationale, I thought this would be a good thing. The shepherd could write just the exceptional information, normally nothing. But when I looked at the template, that wasn't it at all. I'd still have to copy the abstract from the document, as an example.

I do understand that someone has to click on "Yes, I *did* check the ID-nits". That's a good thing. Explaining the consensus process background is good, too. That's real information. And I really do understand the nasty but necessary has-the-IPR-been-checked question. But could we get rid of the rest? The abstract is in the document. The shepherd is in the source address of the e-mail, or the userid of the person using the tracker.

Jari

---

From: Loa Andersson <[email-masked]>
Date: Tue, 25 Sep 2012 20:11:24 +0200

jari,

now we are practically back to how it was before the the proto-write-up
project :), well the things u needed to do was a little bit different,
but number of things were about the same.

/Loa

On 2012-09-25 20:07, Jari Arkko wrote:
> I'm with Bob, too.
>
> But I have a slightly different, additional, angle. When I saw the
> e-mail, and the rationale, I thought this would be a good thing. The
> shepherd could write just the exceptional information, normally nothing.
> But when I looked at the template, that wasn't it at all. I'd still have
> to copy the abstract from the document, as an example.
>
> I do understand that someone has to click on "Yes, I *did* check the
> ID-nits". That's a good thing. Explaining the consensus process
> background is good, too. That's real information. And I really do
> understand the nasty but necessary has-the-IPR-been-checked question.
> But could we get rid of the rest? The abstract is in the document. The
> shepherd is in the source address of the e-mail, or the userid of the
> person using the tracker.
>
> Jari
>

-- 

Loa Andersson                         email: [email-masked]
Sr Strategy and Standards Manager            [email-masked]
Ericsson Inc                          phone: +46 10 717 52 13
                                              +46 767 72 92 13

---

From: Barry Leiba <[email-masked]>
Date: Tue, 25 Sep 2012 17:26:40 -0400

>    http://tools.ietf.org/group/iesg/trac/wiki/DraftShepherdWriteupWgAlternate

Thanks, everyone who commented, for the feedback.  Most of the
comments are saying that you think this format will present extra work
for no real benefit, and that the Q&A format was easy to respond to.
You wonder what the problem with that was.

My view (I'm not speaking for the IESG as a whole here) is that the
problem we're trying to solve is that the shepherd writeups give us a
lot of irrelevant stuff we don't need (there's little value in a
string of "yes, no, no, yes, n/a, yes" that's really just saying that
this document in unremarkable in these regards), and doesn't
sufficiently encourage some explanatory prose chatting when
explanation is what's needed.  The point of the change, then, is that
the shepherd can go through the checklist and just say, "Yep.  Yep.
Yep.  Yep.  Ah, right, I have to check on that one!", and/or "Yep.
Yep.  Yep.  Yep.  Ah, here's something I need to explain."  And then
what you actually write up and send can focus on the explanations we
need.

To demonstrate what I'm getting at, I've taken two writeups from this
week's telechat and done an example re-write to show how they might
have been done with this proposal.  The two documents I've picked (one
from NEA and one from KRB) are different in the complexity of the
writeup that's needed, and the originals were written by shepherds
with different styles -- both of the originals were perfectly fine
writeups, and I think both of the re-writes would have been just as
easy for them to write.  The rewrites use text copied from the actual
writeups, but are significantly trimmed,  Have a look, in the examples
at the top of the proposal:

The NEA writeup, especially, is very concise, because most of Susan's
original was spent in saying that this question or that question
wasn't applicable or had a perfectly normal answer.

--------------

Dan and Jari bring up a different point (perhaps as former ADs
themselves): how this relates to the announcement writeup.  Jari
complains about the silliness of asking the shepherd to paste in the
abstract and to tell the responsible AD who the shepherd and
responsible AD are.  The answer, of course, is that the current
writeup basically asks the shepherd to write the approval announcement
for us.  Dan wonders if that's going away, and now the AD will have to
sort this out.

My thought is that it's a combination of a change in the announcement
format and that yes, the AD will have to pick text out for it a bit
more.  I see the approval announcement template changing, to merge the
"Working Group Summary" and "Document Quality" portions.  The
"Technical Summary" part would come from section 1 of the new writeup,
while the merged part would come from section 2... but it might often
not be the whole section copied and pasted -- the AD will often have
to select which text to put in there.  In practice, I find that I
often want to edit the text that goes in there anyway, so it's often
not just a copy/paste anyway.

But Jari's point is well taken, and I'll bring the question back to
the IESG: The third question in the proposed checklist already asks if
the abstract is good, so can't the AD just copy/paste the abstract
from the document, and not require the shepherd to put it into the
writeup?  And, of course, as Jari says, there's little reason for the
shepherd to have to say, "I'm the shepherd and you're the AD."  We
need to put that in the approval announcement, but there's no reason
you should have to put it into the writeup.

--------------

Again, thanks for the feedback so far.  If there are further comments
after you look at the NEA and KRB samples I've added, please send it
on.

Barry

---

From: "Black, David" <[email-masked]>
Date: Tue, 25 Sep 2012 18:16:13 -0400

Barry,

There may be two related concerns here ...

> what you actually write up and send can focus on the explanations we

I understand that there's limited value to the IESG in reading a list
of expected answers.  OTOH, I think there's value to the IETF in
ensuring that the shepherd checked every one of the process concerns
represented by that list of questions.  My recollection is that many
of the questions for the shepherd writeup came from a checklist that
was posted on the IETF web site - I find myself doing more in the way
of process checks for the questionnaire than I did against that posted
list because I have to answer every question.  I plead guilty to using
the questionnaire as a crutch to make sure that I do what needs to be
done ... but it's an *effective* crutch :-).

In contrast, it might help focus shepherd and IESG attention on the
explanations if they were factored out of the list of questions
that have routine answers.  For example, the questionnaire could be
restructured so that the list of routine questions is introduced
with something like:

	The following list of questions have expected answers in
	[square brackets].  Each question should be answered explicitly,
	possibly with N/A indicating that the question is not
	applicable to this draft.  If any applicable question has
	an answer that is not an expected answer, please explain
	here:

Thanks,
--David

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Barry Leiba
> Sent: Tuesday, September 25, 2012 5:27 PM
> To: [email-masked]

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 26 Sep 2012 10:00:47 +0900

On 2012/09/26 7:16, Black, David wrote:

> represented by that list of questions.

So the shepherd should check all items, but the IESG wants to see only 
the exceptional ones.

Well, how do we usually deal with such a discrepancy? Why not try to 
create a tool that does that for us?

A web form with a bit of logic behind would probably go a long way; the 
only disadvantages I can see now is that writing longer texts (paragraph 
and up) in a web form isn't very convenient, and that it's not trivial 
to make it possible to work offline.

On the other hand, it would be easy to have buttons saying [need more 
info here] for the AD to press and send the thing back to the shepherd 
to get further explanations. Links to tools like id-nits (or even 
embedded results),... also are possible.

Just an idea.

Regards,    Martin.

---

From: Barry Leiba <[email-masked]>
Date: Wed, 26 Sep 2012 12:30:45 -0400

David says...
> represented by that list of questions.

I trust the chairs (and the non-chair shepherds they delegate to) to
check the checklist; I don't need to have it handed in, in order to be
confident of that.  And, of course, if an AD wants to ask her
chairs/shepherds to turn in the completed checklist, she can do that.
In either case, the checklist -- your "effective crutch" is there for
you.

Martin says...
> So the shepherd should check all items, but the IESG wants to see only the
> exceptional ones.
>
> Well, how do we usually deal with such a discrepancy? Why not try to create
> a tool that does that for us?
>
> A web form with a bit of logic behind would probably go a long way

Yes, and I'm sorry I didn't mention this: I had passed this by a few
chairs for comment before putting it out publicly, and a web form was
suggested.  I'm not sure we'd want to require the use of a web form
(most significant issues are being able to compose it offline, and
being able to start it, go away, and come back; they can both be dealt
with, though), but having a web form to help the process might be
good.

Barry

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 26 Sep 2012 14:30:01 -0400

> with, though), but having a web form to help the process might be

Yeah; I have concerns about this.  I keep my writeups in a git
repository, along with reviews and other related materials.  They
sometimes get modified over extended periods, and often are started well
before I'm ready to send a publication request.  This sort of work in
progress is something I'd prefer to hold on to myself, rather than
putting it in some web service that I can't access offline or easily
edit.

-- Jeff

---

From: Loa Andersson <[email-masked]>
Date: Wed, 26 Sep 2012 20:46:48 +0200

Folks,

I agree with Jeff! The write-up is work done over some and sometimes
changed, added to and extended several times, I prefer to have the
revision control.

To me it seems like that it is not entirely clear what is broken.

/Loa

--

---

From: Mary Barnes <[email-masked]>
Date: Wed, 26 Sep 2012 14:01:59 -0500

I agree.  I like the form and the inline reminders to double check things.
 If the issue is that some folks are not putting in enough information, I
think the AD should work with the shepherd to improve the content.  I don't
think a different form for the write-up will necessarily fix the problem of
shepherds not being thorough enough.  If the ADs only want specific
information then write a tool that parses the text to include only the
information the AD is interested in.

Mary.