Subject: Re: Re-revised idea for early cross-area review

From: Allison Mankin <[email-masked]>
Date: Wed, 15 Feb 2006 19:29:44 -0800

> 
> >
> >Q: what happens when the number of review requests exceed the number of 
> >reviewers,
> >or a review is taking a really long time?  (How long is a really long 
> >time in this case?)
> >
> >Do WGs proceed anyway with draft-01 and beyond?
> >Or wait until their draft-00 review is finished before proceeding?
> >
> 
> The dispatcher finds another reviewer and downgrades the delinquent 
> assignee.  Russ and I kept track of which security directorate members 
> didn't pull their weight (and I assume that he and Sam still do); we 
> found it essential to getting things done.
> 

I don't see these reviews as critical-path.  The working group
should keep working, and not block or change their activities
in any way while a review is pending.  As Brian's draft says,
once a review has come out, then there is a natural working group action:

> - WGs are expected to treat these generalist reviews seriously, especially
>    when they indicate serious cross-area problems or gaps. But their formal
>    place in the IETF process is the same as any other comments.

This means that the review receives a written response to its points, on
the mailing list.  Discussion and working group consensus would determine
any next steps, as in normal comments.  This is different from late
evalation reviews.

The main reason for having the dispatcher is to be sure that broad
coverage is occurring.  If an -01 were to show up quickly before the
review showed up, then the review could be on the -01.  "Early" 
doesn't have to be a mathematically precise term.

Allison

---

From: Jari Arkko <[email-masked]>
Date: Thu, 16 Feb 2006 09:52:56 +0200

Allison Mankin wrote:

> in any way while a review is pending.
>
Right. I don't think we should set this up as a guaranteed
service with full coverage of all -00 drafts and committed
deadlines. In practise, we can hope to cover the most important
stuff. And yes, WGs should keep on working.

> doesn't have to be a mathematically precise term.
Yes.

--Jari

---

From: Andy Bierman <[email-masked]>
Date: Fri, 17 Feb 2006 07:51:01 -0800

> stuff. And yes, WGs should keep on working.

This approach should probably work fine for this type of review.
It would be a disaster for detailed final reviews, of course, but
it is now clear that not what this is about.

IMO, more energy should go into proactive measures in the
form of detailed guidelines RFCs for every aspect of the
specification.  If  I'm supposed to document my use of TLS
in a specific detailed manner, then write it down in an RFC
so I can get it right the first time.

Document reviewing is an inherently reactive process.
The thing people hate the most about writing MIBs
are the 532 crappy little rules you have to follow to get it "right".
Only the MIB Doctors know all the CLRs, and they won't
tell you until the Review.  (Now we have RFC 4181, so authors
can find out all the rules in advance. They will still hate MIBs ;-)

The CLR factor for RFCs is way worse than for MIBs.
Since they don't seem to be going away, they have to be documented.

>> doesn't have to be a mathematically precise term.

Of course you have to have a dispatcher.
Review dispatching is an art-form in itself, if you care about matching
reviewer technical skill sets with review material, and also keep the
reviews moving through the pipe.  But that's still the easy part.

> --Jari

Andy

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 21 Feb 2006 19:09:14 +0100

Andy,

I think that has clarified the proposal - sorry if it wasn't
clear enough in my note. I'm encouraged enough to think
about a full length description (or finding a volunteer to
write such...).

I agree with you about encouraging guidelines documents;
at the least they help a reviewer to explain basic problems
in an early draft.

     Brian

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2006 09:47:18 -0800

> at the least they help a reviewer to explain basic problems

It helps authors get it right the first time, which I think is critical.
It also helps the reviewer if they can say
"Go read RFC xxxx, and redo section Z according to these guidelines."
when the early draft is too rough to bother reviewing yet.

Andy