Subject: Early External Review process experiments

From: Lisa Dusseault <[email-masked]>
Date: Mon, 18 Dec 2006 12:02:07 -0800

Early External Review is a doc process elaboration some of us have  
been playing around with, to make it easier to find external  
reviewers where desirable:
	- To try to uncover serious problems before IETF last call or IESG  
Evaluation.  Delays and changes at this stage are frustrating.
	- To expose new concepts to people outside the WG as a sanity check  
and as education.  Review teams share reviews amongst themselves so  
it's not just one person per review exposed to a new concept.
	- To get more detailed review of specific areas where the WG doesn't  
have deep expertise: for example, XML, i18n, manageability, security  
aspects.

If an expert/outsider review is available before WG last call, a WG  
editor ought to be able to work in changes  without requiring a new  
sequential step in the process to delay everything.

The Security Directorate review team is perhaps the best example of  
this kind of review so far, when those reviews happen early (GenArt  
reviews aren't typically early).  There's no guarantee that a  
Security Directorate review will find all the problems that Russ and  
Sam might find, nor that the review will be limited to only IESG- 
Evaluation-blocking problems, but some WGs have found it helpful to  
get the scrutiny of a security expert before finishing a document.

In addition to the SecDir team, there's an Apps expert review team  
and a MIB doctor team which may also be able to look at  
manageability.  Other teams may get created in the future and added  
to the list.

Early review, ideally, does not mean finding nits, because we  
shouldn't expect documents to be finished and polished when they're  
submitted for early external review.  It means reviewing  
architectural and technological choices and raising high-level  
issues.  Of course an early reviewer may point out unclear parts of  
the document or missing references, but hopefully the document allows  
an external reviewer to understand the proposal in moderate detail.

If you're interested in participating in this experiment, as WG  
chairs you're the ones to submit documents.  You can submit a  
document just before WG last call, or when questions about certain  
technology choices are being raised and no solid answers emerge from  
the WG, or any other appropriate time.

More information, addresses of review teams, and some tracking  
information I hope to maintain, can be found on this Wiki page:  
http://www1.tools.ietf.org/group/iesg/trac/wiki/EarlyExternalReview

I can act as a tracker and/or review submitter for now, and plan to  
keep the status table on the Wiki page up-to-date with all the review  
status I'm given (unless and until the experiment is an overwhelming  
success and I can't keep up).  The status information is intended to  
allow us to evaluate later how often this is used, and perhaps even  
how successful it is based on how well a document does after going  
through early external review.

Other comments welcome,
Lisa