Subject: WG Chair draft reviews

From: "Murphy, Sandra" <[email-masked]>
Date: Fri, 20 Sep 2013 15:33:44 +0000

The WG states mentioned in 6174 (and thus the datatracker "change state" page) indicate a revised ID might be needed on the basis of wglc, AD review or IESG review.  But not on the basis of a wg chair review.

I have seen wg chairs in some wg do their own distinct review of a draft before requesting publication.  Of course, the wg chair needs to be sure the comments received in wglc are adequately addressed; I'm talking about a distinct wg chair review for document quality.

I would think the wg chairs have a responsibility more than the other members of the wg to ensure document quality.  But the WG states of 6174 might be taken to imply that they do not.

What do other chairs think?  Do you do you own review, other than as a wg member in wglc?

Note that the document shepherd writeup requires that the shepherd must do his/her own review.  So the shepherds have a distinct review responsibility but the chairs do not?

--Sandy

(It's been at least a week since we've had the last wg chair 100+ message chain and I'm feeling the loss.  :-)  )

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 20 Sep 2013 15:58:06 +0000

> The WG states mentioned in 6174 (and thus the datatracker "change state" page) indicate a revised ID might be needed on the basis of wglc, AD review or IESG review.  But not on the basis of a wg ch…

IMHO, that's a judgement call, not a legal one. If the changes required by the IESG discussion are trivial, they can be edited in by the RFC Editor given author/WG agreement; changes that are more substantive, regardless of the timing in the review process, call for an updated document. If the WG Chair makes substantive comments, from my perspective, they are equivalent to WGLC comments. The next review can't really happen without the update (unless you're willing to say "please review this, but understand that sections 1, 2, 3, 4, and 5 need to be completely rewritten" or whatever). SO you ask for the update.

My two yen

> (It's been at least a week since we've had the last wg chair 100+ message chain and I'm feeling the loss.  :-)  )

----------------------------------------------------
The ignorance of how to use new knowledge stockpiles exponentially. 
   - Marshall McLuhan

---

From: "Bernie Volz (volz)" <[email-masked]>
Date: Fri, 20 Sep 2013 16:07:35 +0000

BTW - Replace "WG chair" with document shepherd (though in most cases they are the same). At least in the case of the DHC WG, there are plenty of times where as part of the shepherding review process, the shepherd has had to request updates. Typically we have done this either as part of the last call or just afterwards (it mostly depends on whether the WG last call had comments requiring other updates -- as then the shepherd should wait for those unless they are just typos).

I agree with Fred that these are close to "normal" WGLC type comments (though they may occur after the call has ended). They carry a bit more weight as otherwise the shepherd may indicate that he/she doesn't believe the document is ready; or ready with exceptions (which the AD or IESG would then need to consider).

- Bernie

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Fred Baker (fred)
Sent: Friday, September 20, 2013 11:58 AM
To: Murphy, Sandra
Cc: [email-masked]
Subject: Re: WG Chair draft reviews 

My two yen

---

From: Dave Crocker <[email-masked]>
Date: Fri, 20 Sep 2013 09:21:18 -0700

On 9/20/2013 8:33 AM, Murphy, Sandra wrote:
>   have seen wg chairs in some wg do their own distinct review of a draft before requesting publication.  Of course, the wg chair needs to be sure the comments received in wglc are adequately addressed; I'm talking about a distinct wg chair review for document quality.
>

Creating a specific state for this essentially means that a wg chair 
review has special status, over and above anyone else's review.

I think that chair and shepherd reviews are a good idea -- and even 
essential for at least one of them -- but I guess I'd question either of 
their reviews being given special status over any other review done 
during WG Last Call.

That is, I think the cleaner and simpler model is to have WGLC cover 
reviews, including by chair and/or shepherd.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Benoit Claise <[email-masked]>
Date: Fri, 20 Sep 2013 18:32:32 +0200

On 20/09/2013 18:21, Dave Crocker wrote:
>>   have seen wg chairs in some wg do their own distinct review of a 
>> draft before requesting publication. Of course, the wg chair needs to 
>> be sure the comments received in wglc are adequately addressed; I'm 
>> talking about a distinct wg chair review for document quality.
>>
>> I would think the wg chairs have a responsibility more than the other 
>> members of the wg to ensure document quality.  But the WG states of 
>> 6174 might be taken to imply that they do not.
>>
> essential for at least one of them -- but I guess I'd question either 
> of their reviews being given special status over any other review done 
> during WG Last Call.
Agreed. No need for an extra state.
A simple email from the doc. shepherd to the authors "please produce a 
new version, because you missed feedback X" is enough.
Anyway, this is the doc. shepherd who tells the AD when the draft is ready.

Regards, Benoit

---

From: "Murphy, Sandra" <[email-masked]>
Date: Fri, 20 Sep 2013 17:00:56 +0000

>Agreed. No need for an extra state.

Extra state - not my implication

WG comment not addressed - obviously that's a chair responsibility

Extra quality responsibility - more my focus.

>> reviews, including by chair and/or shepherd.

Cleaner still to have the ADs join in the wglc, to be considered just as any other wg member?  :-)

I think what this is devolving to is a question of timing.  Chair comments during wglc is just less delay.

--Sandy

---

From: Dave Crocker <[email-masked]>
Date: Fri, 20 Sep 2013 10:10:36 -0700

> Extra quality responsibility - more my focus.

Given the separate thread discussion elsewhere, about possible backdoors 
in IETF protocols, I think it's certainly worth considering extra 
quality assurance, but wonder whether the wg chair is the type of review 
to institutionalize for this.  Probably not.

> Cleaner still to have the ADs join in the wglc, to be considered just as any other wg member?  :-)

Interesting you should mention that.  So forgive me for ignoring the 
smiley and taking the query seriously...

I think the strategic model that ADs have to do deep reviews is 
somewhere between bizarre and silly.  In organizational terms, these are 
second-level (or even third-level) managers, coming in at the end of the 
process.  In terms of a serious, strategic quality assurance mechanism, 
that model is neither technically reasonable nor cost-effective.  It's 
really quite remarkable, just how expensive this model is.  And for all 
of the useful review points the ADs come up with, I seriously doubt we 
could document a reliable pattern of major benefit.  (By major benefit, 
I don't mean "the documents get better", because I'm quite sure they do 
get better.  By major benefit, I mean "a serious flaw was found or 
fundamental improvement was developed, as compared against all of the 
major flaws that sneak through to publication.  Cost/benefit at an 
overall systems level.)

It's not a question of whether the ADs attempt good reviewing; it's 
whether its reasonable to have them do it as a regular part of their 
work.  I think it's not.

So frankly, I think making ADs reviews optional, and moving them to WGLC 
would be quite a good thing.

Note that I'm not suggesting a change to the vote/discuss model, merely 
the kind of work expected from all ADs within that model.

> I think what this is devolving to is a question of timing.  Chair comments during wglc is just less delay.

Hmmm.  It might actually be a case of that tactical point turning out to 
have some significant strategic benefit.

d/

---

From: Loa Andersson <[email-masked]>
Date: Mon, 23 Sep 2013 13:48:20 +0800

> I think what this is devolving to is a question of timing.  Chair comments during wglc is just less delay.

All,

I was a bit confused about this discussion. The way it works for me as
a chair is that I review a document at least three times, at these
reviews  I might produce slightly different type of comments. None of
these reviews are really formalized.

- I review a document before sending out a poll to see it we have
   consensus to make it a working group document. Looking mostly at
   issues like is this for the working group, is it charter and does
   it address a technical problem we want to solve.
   The focus is mostly technical and I don't bother to much about nits
   and if the document are according to ID guidelines.
   At this time we also have a wg review team taking a look at a
   document, answering question like "do we want to do this", "is this
   a good enough starting point".
- review the document in or before working group last call, I tend to
   do this review before the wglc if I will send out the wglc. To make
   sure that I think the document is ready for wglc, and if another chair
   starts the wglc I tend to do the review as part of the wglc.
- I review the document when preparing to send the "publication
   request", making sure that both the authors, wg chairs and shepherds
   have done what is possible to have a good document; this is the point
   where I do the final nit-hunting, but also make sure that I'm
   comfortable with e.g. IPR disclosures and the level of overall review.

In the period between the first and second review there is the normal
working group process going on, nailing down the solution and having
the working group converging on that solution. This also when I'd call
for expert/third party reviews.

I'm not saying that we should formalize this, but is fairly convinced
that the amount of review that working chairs, shepherd, third  party
experts and the wg in general put into a document the better the output
is.

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64