Subject: [Gen-art] Adjusting review guidelines slightly

From: Brian E Carpenter <[email-masked]>
Date: Wed, 07 Dec 2005 15:49:33 +0100

Hi again,

Here's a suggested update of our review guidelines. (Mary, the changes
are bracketed by <font color="red">...</font>, plus I fixed two
URLs (the NITS and BCP26 references).

Only three changes of significance:

1. About LC reviews. We don't currently follow normal IETF process, which is that
LC comments are supposed to be sent to the ietf (or iesg) lists. I've added
to the guidelines:

Since IETF Last Call comments are commonly sent to the IETF discussion list,
the reviewer may also choose to do that; in any case the review will be public.

2. About the routing of IESG-agenda reviews. They really do need to be sent to
the interested parties unless they're no-ops. And authors really need
to see the FAQ. So I have changed it to:

Reviews should be copied to authors, responsible ADs and WG chairs,
unless the review finds no issues. Also, including a link to the FAQ
in the email has proved essential for the recipients of the review in
understanding the context, especially since Gen-ART review is a
new experience for the average author.

3. Dialling back the editorial review. I've added

However, a detailed (copy-editing) editorial review is not productive; if
the reviewer feels that a draft is too badly written to advance, it will
be sufficient to say so with one or two examples.

     Brian
Guidelines for review - the General Area Review Team (GEN-ART)

These are the current (December 2005) draft guidelines for late
review in the GEN-ART.

The guidelines focus on the model that has proved fairly
productive in the Ops Directorate: Quick review close to telechat
time, to advise the AD on issues that remain serious.

See also: FAQ,
Reviewer
list, Summary
of Reviews

Timeline of review

Review is either done at Last Call, when the document appears on
the IESG agenda, or both.

The process for reviewing documents at Last Call:

The Secretary assigns a reviewer
around the time of Last Call announcements – typically
waiting until a batch of at least 4 is available for review OR more
frequently if the LC date is within one week.

We expect the reviewer to be done
before the end of Last Call.

The reviewer sends the review to
the Gen-ART list

The reviewer also sends the review (or a pointer to it) to
AD, WG chairs & authors, and the WG if it's a WG document (and
the WG list accepts mail). Since IETF Last Call comments are
commonly sent to the IETF discussion list, the reviewer may also
choose to do that; in any case the review will be public.

The process for reviewing documents when they appear on the IESG
agenda:

Secretary checks the IESG agenda
one week before the IESG telechat. The agenda is typically
finalized Thursday evening EDT, with late agenda items sometimes
being added on Friday morning, thus the assignments are out either
late Thursday evening or early Friday morning (EDT).

For documents reviewed at Last
Call, a new review is only asked for if the document is revised or
issues resolved.

The Secretary names a reviewer per
document, more or less randomly. For documents that have been
reviewed before, the same reviewer is kept, if that person is still
available.

Reviewers send their review to the
Gen-ART list no later than the day before the telechat (i.e., 2 PM
EDT on Wednesday) (earlier is better!) There's no penalty for
handing in reviews you didn't promise to do :-)

Reviews should be copied to authors,
responsible ADs and WG chairs, unless the review finds no issues.
Also, including a link to the FAQ
in the email has proved essential for the recipients of the review in
understanding the context, especially since Gen-ART review is a new experience
for the average author.

If the AD concludes that the concerns raised by the reviewer
warrant blocking the document, the AD will do so.

The telechats are every other Thursday, with the first telechat
for 2006 on January 5th.

Except for skipping a telechat around
the IETF meetings or Holidays, these rarely change.

Form of review

Rather than invent new guidelines, this document steals liberally
from draft-carpenter-solutions-sirs-01, and adapts for the special
"late, quick review" case and the General area's questions.

Each review must start with a summary statement chosen from or
adapted from the following list:

This draft is ready for
publication as a [type] RFC, where [type] is Informational,
Experimental, etc. (In some cases, the review might recommend
publication as a different [type] than requested by the author.)

This draft is basically ready for
publication, but has nits that should be fixed before publication.

This draft is on the right track
but has open issues, described in the review.

This draft has serious issues,
described in the review, and needs to be rethought.

This draft has very fundamental
issues, described in the review, and further work is not
recommended.

Unfortunately, I don't have the expertise to review this
draft.

The length of a review will vary greatly according to
circumstances, and it is acceptable for purely editorial comments to
be sent privately if it's obvious that the document will have to be
substantially revised anyway. All substantive comments must be
included in the public review. Wherever possible, they should be
written as suggestions for improvement rather than as simple
criticism. Explicit references to prior work and prior IETF
discussion should be given.

Reviewers should review for all kinds of problems, from basic
architectural or security issues, Internet-wide impact, technical
nits, problems of form and format (such as IANA Considerations or
incorrect references),and editorial issues. Since these reviews are
on documents that are supposed to be finished, the review should
consider "no issue too small" - but cover the whole range
from the general architectural level to the editorial level.
However, a detailed (copy-editing) editorial review is not
productive; if the reviewer feels that a draft is too badly
written to advance, it will be sufficient to say so with one or
two examples.

The review should apply generally agreed IETF criteria, such as

[RFC1958]
The Architectural Principles of the Internet

[RFC3426]
General Architectural and Policy Considerations

[RFC3439]
Some Internet Architectural Guidelines and Philosophy

[NITS]
The "I-D Nits" document maintained by the IESG

[RFC2223]
Instructions to RFC Authors

[BCP26]
Guidelines for Writing an IANA Considerations Section in RFCs

[RFC3552]
Guidelines for Writing RFC Text on Security Considerations

as well as any other applicable architectural or procedural
documents. It is important that reviews give precise references to
such criteria when relevant.

Of special interest to the GEN area (because it's no area's
special interest) is:

Clear description of why the
document or protocol is useful to the Internet

Adherence to IETF formalities such
as capitalized MUST/SHOULD (ID-Nits)

Useful and reasonable IANA
considerations

Correct dependencies for normative
references

That it's written in reasonably clear English

Mailing List

All reviews are posted on the IETF gen-art
mailing list.

Archiving of reviews

All reviews are also archived and publicly visible. The archive is
here.

_______________________________________________
Gen-art mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/gen-art