Subject: RE: WG chair use of the ID-tracker: Proposed tool additions

From: "Romascanu, Dan \(Dan\)" <[email-masked]>
Date: Mon, 15 May 2006 12:01:51 +0300

I would agree with Henrik. As the AD in charge with dispatching MIB
review requests a state telling me that the WG is waiting for a MIB
review would be useful, one form or another. 

MIB reviews are of course not the only (in)famous examples. Security
reviews are another examples. MIB documents are in many cases needing
reviews from interface technology experts. And so on. 

Dan

> -----Original Message-----
> From: Henrik Levkowetz [mailto:[email-masked]] 
> Sent: Monday, May 15, 2006 10:53 AM
> To: Eliot Lear
> Cc: 'Working Group Chairs'
> Subject: Re: WG chair use of the ID-tracker: Proposed tool additions
> 
> Hi Eliot,
> 
> on 2006-05-15 08:42 Eliot Lear said the following:
> > Hi Henrik,
> > 
> > I have some confusion about the states.  Is there sufficient 
> > difference between WG Last Call & Awaiting Reviews to 
> require a separate state?
> 
> During the last year or so in MIP4 we've had two documents 
> where we for different reasons have asked for reviews, 
> without it being appropriate to do a WGLC.  Allison also had 
> examples and saw a clear difference between these.  The most 
> obvious cases are probably those needing MIB doctor review or 
> similar, but there are other cases too, as far as I can see.
> 
> >  To
> > my line of thinking WG last call is the shot across the bow 
> that gets 
> > otherwise disinterested people to actually read the doc.
> 
> Yah.  (And sometimes even that doesn't work ,;-)
> 
> >  I have no
> > problem with a proto checklist in the tool asking how many reviews 
> > were don and even by whom.
> 
> Ok.
> 
> 
> Regards,
> 
> 	Henrik
> 
> 
>

---

From: "Romascanu, Dan \(Dan\)" <[email-masked]>
Date: Wed, 17 May 2006 10:05:40 +0300

I do not hate this really, I even agree to a certain extent, because
disagreeing with facts of life is not a good approach, but then, at what
authors and WGs consider to be hour 23:45, discovering basic
architecture flaws (like in security or MIB design) is rather traumatic,
isn't it? 

Dan

> From: Eliot Lear [mailto:[email-masked]] 
> Sent: Tuesday, May 16, 2006 9:43 PM
> To: Brian E Carpenter
> Cc: 'Working Group Chairs'; [email-masked]
> I know you guys hate this, but in my opinion IETF last call 
> is cross area review.
> 
> Eliot
> 
> 
>

---

From: "Romascanu, Dan \(Dan\)" <[email-masked]>
Date: Wed, 17 May 2006 10:12:28 +0300

Well, yes, sending the request for MIB Doctor review after the WGLC
seems to be the current procedure. However, the MIB Doctors team is
discussing ways to improve the process, and one of the current proposals
is to have MIB advisors allocated per WG, so that they can provide
advice in the earlier phases of data modeling and MIB design and catch
more of the possible design and architecture issues as early as
possible. The later review would then deal only with SMI and
documentation issues.

Dan

> From: Allison Mankin [mailto:[email-masked]] 
> Sent: Monday, May 15, 2006 8:11 PM
> To: Eliot Lear

> 
> I just noticed a bug in our text - I'll ask Dan Romascanu to 
> confirm this, but I think that often a WG is advised to ask 
> for MIB Doctor Review after a WGLC, so that the MIB Doctor is 
> working with the WG's finished text, and the WG just reacts 
> to that review and then requests publication from those 
> review points.  I think the same is probably true of formal 
> Security Review.
>

---

From: "Romascanu, Dan \(Dan\)" <[email-masked]>
Date: Wed, 17 May 2006 12:13:22 +0300

The additional possibility would not work, as at this point in time it
is the WG chairs who hold the stick. 

Dan

> Sent: Wednesday, May 17, 2006 12:06 PM
> To: Romascanu, Dan (Dan)
> Cc: Brian E Carpenter; Allison Mankin; Working Group Chairs
> Dan,
> One possibility is to send the announcement of WGLC to the 
> IETF-Announce list.  One additional possibility is for the AD 
> to block document advancement on external review when he/she 
> sees fit.  It'd fit the "I'll trust you until..." model.
> 
> Eliot
>

---

From: [email-masked]
Date: Wed, 17 May 2006 16:03:20 -0400

Brian,

I understand the desirability of early MIB reviews, and I've lived
through an example of the MIB review you never wanted to see
(fortunately, not as WG chair), so it's not completely hypothetical
(and it was not fun at the time) ...

The important point here is that the WG MIB Expert and OPS Area MIB
Doctor reviews are different from a process standpoint, and hence
the draft being discussed probably should not have used the phrase
"MIB Doctor".  I rely on the WG MIB Expert to (re-)check the MIB before
publication - his/her final review comments will usually show up as part
of WG Last Call (and s/he has usually been working with the authors prior
to that).  In addition, there's also an OPS Area MIB Doctor review that
currently happens after publication is requested (and is covered by the
existing Expert Review state in the I-D tracker) - while the OPS Area
reviews take time, they are an important quality control measure that
is relied upon to ensure MIB quality in published RFCs.

One WG state that I need appears to be missing: "WG Document - WG Last
Call Complete" to cover a draft that has completed WGLC successfully,
but for which publication hasn't been requested yet (e.g., overworked
WG chair hasn't gotten the(a) "round tuit" for the PROTO writeup, the
WG MIB Expert needs to re-check something, it's waiting for another draft
that will be along soon, etc.).  This state also needs a "Revised ID
Needed" substate (e.g., idnits run as part of preparing the PROTO
writeup turns up something major that should have been caught earlier,
but wasn't).  Calling this state "Parked" doesn't work, because the
document can generally be worked on, whereas "Parked" means "stuck",
potentially long-term (e.g., a "WG Last Call Complete" document may
get "Parked" if the editor vanishes).

Thanks
--David (imss, ips, rddp WG chair)
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

> From: Brian E Carpenter [mailto:[email-masked]] 
> Sent: Wednesday, May 17, 2006 12:28 PM
> To: David Harrington
> Dave,
> > I like to make a clear distinction between being a MIB technical
> > advisor and being a MIB Doctor. The first is for early reviews and
> > sporadic advice; this is really just a design review role. A MIB
> > Doctor review is an incredibly thorough, manual review of a document
> > submitted for advancement on the standards track. 
> 
> I understand your issue, but let me challenge you with a hypothetical
> review after WGLC that I never want to see:
> 
> "This MIB is badly conceived and needs to be completely reformulated."
> 
> The whole early review debate is about how to avoid that happening
> late in the process. Your points are well taken, but we need to
> empower WG Chairs to get initial reviews done earlier, to get that
> sort of alarm signal. That is not the same as asking for complete
> MIB Doctor review at the -00 stage. Are we on the same wavelength?
> 
>     Brian
>

---

From: "Romascanu, Dan \(Dan\)" <[email-masked]>
Date: Thu, 18 May 2006 09:49:49 +0300

I did not expect this to pass easy :-)  But yes, the text seems good to
me and it's progress in the direction that I feel it's needed. 

Dan

> Sent: Wednesday, May 17, 2006 5:05 PM
> To: Henrik Levkowetz
> Cc: Romascanu, Dan (Dan); Working Group Chairs; Brian E Carpenter
> Hi,
>   Commenting on my own proposal...
> 
> I've received some private comments and had some discussion 
> which makes me think that it's probably a good idea to keep 
> WGLC as a fairly pure state, and not mix it with the review 
> state.  This doesn't diminish the importance of getting all 
> kinds of comments, including comments resulting from 
> cross-area review, at WGLC time, but it seems better to keep 
> the "Awaiting <whatever>"
> sub-states for the explicit "WG Document Awaiting Reviews" state.
> 
> So I'm inclined to withdraw the suggestion below, only 
> keeping the last bullet, with a changed wording:
> 
>  * Permit multiple parallel sub-states, so it is possible to
>    indicate e.g., for a document in "WG Document Awaiting
>    Reviews", that 4 reviews are required, and also that the
>    document is awaiting cross-area review and MIB review.
> 
> 
> Regards,
> 
> 	Henrik
> 
> on 2006-05-17 13:01 Henrik Levkowetz said the following:
> > Hi,
> > 
> > on 2006-05-17 09:16 Romascanu, Dan (Dan) said the following:
> >> With the risk of playing a broken record, I would 
> reiterate the idea 
> >> that cross-area expert reviews need not be separate 
> states, but can 
> >> rather be part of the WGLC. This asks more coordination 
> between the 
> >> WG chairs and the expert teams, but would avoid inserting 
> >> process-blocking states before or after WGLC.
> > 
> > 
> > Ok, so would the following work:
> > 
> > Change the draft to:
> > 
> >  * Add "In WG Last Call" to the possible next states for the
> >    "Active WG Document" state
> > 
> >  * Let the sub-states of the "WG Document Awaiting Reviews"
> >    state also be possible sub-states of "In WG Last Call"
> > 
> >  * Permit multiple parallel sub-states, so it is possible to
> >    indicate e.g., for a document in "In WG Last Call", that
> >    it is the 2nd last call, and also that the document is
> >    awaiting cross-area review
> > 
> > ?
> > 
> > 
> > Regards,
> > 
> > 	Henrik
> > 
> > 
> >> 
> >> Dan
> >>  
> >> 
> >>  
> >>  
> >> 
> >>> Sent: Tuesday, May 16, 2006 4:23 PM
> >>> To: Allison Mankin
> >>> Allison Mankin wrote:
> >>> > Eliot,
> >>> > Your question made me realize that the variation in WG
> >>> practices means
> >>> > we should make the explanation clearer.  Some WGs, eg. DNSEXT, 
> >>> > will not send a document to WGLC till a minimum number of formal
> >>> reviews are done
> >>> > (their current number is five, I believe).    Some 
> groups have 2-3.
> >>> > Others don't use formal reviews.  The state is indeed
> >>> purely elective.
> >>> 
> >>> Just as a data point, the IESG retreat ten days ago reiterated on 
> >>> the importance of early cross-area review,
> >>> *before* WGLC, to limit late surprises. Lisa Dusseault 
> now owns this 
> >>> topic within the IESG, and you will hear more in due 
> course. From a 
> >>> process point of view this is indeed elective, but there are good 
> >>> reasons, that have been discussed on this list in the past, for 
> >>> doing it.
> >>> 
> >>>      Brian
> >>> 
> >>> 
> >>> 
> >> 
> >> 
> > 
> > 
>