Subject: Re: WG chair use of the ID-tracker: Proposed tool additions

From: Allison Mankin <[email-masked]>
Date: Wed, 17 May 2006 07:41:06 -0700

I'd like to weigh in supporting Loa's very well-stated view, from the viewpoint of
the PROTO team's general goals, which is for the WGs/Chairs to take very
strong responsiblity for their documents' quality.  The WG Last Call is "last" - 
to be a real sign off.  So it makes sense for many external reviews (MIB Doctor,
Security, MIME) to be done just before WGLC, close to it, at the mature
document state.

The WG Chairs should assess and meet Loa's criteria as WGLC approaches
(during the WG internal reviews while the document is maturing) and decide
when those external reviews are appropriate.  

That said: we're the IETF.  We should be careful not to get hung up on formality.

So, I think, we in PROTO team should draw you a state diagram at this point, 
and make sure everyone sees where we are.

Allison

> 
> to my mind "WG Last Call" is a point where we have a
> document that is not going through changes, everyone
> has the same document to comment on.
> 
> Outcome can be:
> 
> Ready to move
> 
> Needs update before moving
> 
> Needs update update and the another wg last call
>   - limited to changes, or
>   - on the entire document
> 
> Possibly also: "We need need to do some other review of some
> kind".
> 
> But neither of these are sub-states of "WG last call", if they are
> sub-states at all, I guess WG external reviews motivates state
> of its own, they should be sub-states of "Active document".
> 
> Point here is that "WG Last Call" is time limited, and state
> machine will more complicated than flexible if we start sub-stating
> it.
> 
> 
> /Loa
> 
> Brian E Carpenter wrote:
> > Henrik Levkowetz wrote:
> >> Hi,
> >>
> >> on 2006-05-17 09:16 Romascanu, Dan (Dan) said the following:
> >>
> >>> With the risk of playing a broken record, I would reiterate the idea
> >>> that cross-area expert reviews need not be separate states, but can
> >>> rather be part of the WGLC. This asks more coordination between the WG
> >>> chairs and the expert teams, but would avoid inserting process-blocking
> >>> states before or after WGLC. 
> >>
> >>
> >>
> >> Ok, so would the following work:
> > 
> > Flexibility is good, but too much flexibility can make us unclear
> > what the "normal" procedure is. However, if we can allow for
> > review states at any point in the WG process I think that gives
> > us freedom to find the best practice. Getting review states
> > recognized in the tracker is already a big win IMHO.
> > 
> >     Brian
> >>
> >> Change the draft to:
> >>
> >>  * Add "In WG Last Call" to the possible next states for the
> >>    "Active WG Document" state
> >>
> >>  * Let the sub-states of the "WG Document Awaiting Reviews"
> >>    state also be possible sub-states of "In WG Last Call"
> >>
> >>  * Permit multiple parallel sub-states, so it is possible to
> >>    indicate e.g., for a document in "In WG Last Call", that
> >>    it is the 2nd last call, and also that the document is
> >>    awaiting cross-area review
> >>
> >> ?
> >>
> >>
> >> Regards,
> >>
> >>     Henrik
> >>
> >>
> >>
> >>> Dan
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Brian E Carpenter [mailto:[email-masked]] Sent: Tuesday,
> >>>> May 16, 2006 4:23 PM
> >>>> To: Allison Mankin
> >>>> Cc: 'Working Group Chairs'
> >>>> Subject: Re: WG chair use of the ID-tracker: Proposed tool additions
> >>>>
> >>>> Allison Mankin wrote:
> >>>>
> >>>>> Eliot,
> >>>>>
> >>>>> Your question made me realize that the variation in WG 
> >>>>
> >>>> practices means
> >>>>> we should make the explanation clearer.  Some WGs, eg. DNSEXT, will
> >>>>> not send a document to WGLC till a minimum number of formal 
> >>>>
> >>>> reviews are done
> >>>>
> >>>>> (their current number is five, I believe).    Some groups have 2-3.
> >>>>> Others don't use formal reviews.  The state is indeed 
> >>>>
> >>>> purely elective.
> >>>>
> >>>> Just as a data point, the IESG retreat ten days ago reiterated on
> >>>> the importance of early cross-area review, *before* WGLC, to limit
> >>>> late surprises. Lisa Dusseault now owns this topic within the IESG,
> >>>> and you will hear more in due course. From a process point of view
> >>>> this is indeed elective, but there are good reasons, that have been
> >>>> discussed on this list in the past, for doing it.
> >>>>
> >>>>     Brian
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> > 
> > 
> 
> 
> -- 
> Loa Andersson
> 
> Principal Networking Architect
> Acreo AB                           phone:  +46 8 632 77 14
> Isafjordsgatan 22                  mobile: +46 739 81 21 64
> Kista, Sweden                      email:  [email-masked]
>                                            [email-masked]
>

---

From: "David Harrington" <[email-masked]>
Date: Wed, 17 May 2006 11:52:59 -0400

Hi,

I would like to weigh in as a MIB Doctor, since this could have an
effect on my volunteer time.

Getting ANY review can be difficult for a chair to get, and chairs
declare WGLC because it motivates people to respond. I have been asked
by chairs to perform MIB Doctor review on documents that were nowhere
near ready for WGLC.  

I like to make a clear distinction between being a MIB technical
advisor and being a MIB Doctor. The first is for early reviews and
sporadic advice; this is really just a design review role. A MIB
Doctor review is an incredibly thorough, manual review of a document
submitted for advancement on the standards track. 

There are only about ten active MIB Doctors to review, in detail, all
the MIBs produced by the IETF, performing checks for things that have
not yet been automated. MIB Doctors cannot afford the time to
volunteer to do MIB Doctor reviews repeatedly for half-baked
documents.

I would not like to see pre-WGLC MIB Doctor review made the
requirement. 

If you want the responsibility for expert review to be moved to the
chairs, I think it would make sense to change the process flow to show
the WGLC as being the WG sign-off, then the document is submitted into
the shepherding process, where the chair is responsible for
shepherding the document through expert review before passing it to
the AD and the IESG. But "official" expert review should be done after
WGLC not before. "Unofficial" expert review can happen anytime at the
request of the chair and the willingness of an expert volunteer.

My $.02
dbh

> From: Allison Mankin [mailto:[email-masked]] 
> Sent: Wednesday, May 17, 2006 10:41 AM
> To: Loa Andersson
> Cc: Working Group Chairs; Brian E Carpenter

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 17 May 2006 18:28:12 +0200

Dave,

> submitted for advancement on the standards track. 

I understand your issue, but let me challenge you with a hypothetical
review after WGLC that I never want to see:

"This MIB is badly conceived and needs to be completely reformulated."

The whole early review debate is about how to avoid that happening
late in the process. Your points are well taken, but we need to
empower WG Chairs to get initial reviews done earlier, to get that
sort of alarm signal. That is not the same as asking for complete
MIB Doctor review at the -00 stage. Are we on the same wavelength?

    Brian

---

From: "David Harrington" <[email-masked]>
Date: Wed, 17 May 2006 14:50:18 -0400

Yes. We are on the same wavelength.

A chair can ask for technical advice at any time - and should. That's
why I like to make the distinction between technical advisor and MIB
Doctor roles. Asking for technical advice will (hopefully) prevent the
"This MIB is badly conceived and needs to be completely reformulated."
issue. But an SNMP expert acting in a technical advisory role
(official or unofficial) does not typically do as thorough a review as
when he is performing a MIB Doctor review (at the end of the WG
process/preparing for IESG review/processing).

Personally, I would like to see every area develop their own MIB (or
mgmt) technical advisors (if we continue to expect MIBs (or mgmt
considerations) for most protocols), and keep "the MIB Doctors"
separate concept for post-WGLC expert review. 

dbh 

> From: Brian E Carpenter [mailto:[email-masked]] 
> Sent: Wednesday, May 17, 2006 12:28 PM
> To: David Harrington
> Cc: 'Allison Mankin'; 'Loa Andersson'; 'Working Group Chairs'