Subject: [INDEP] Issues in draft-klensin-rfc-independent-02.txt

From: Brian E Carpenter <[email-masked]>
Date: Fri, 09 Jun 2006 14:00:14 +0200

I generally support this draft and am grateful that John wrote it.
I do believe that the general principle of checks and balances
that the Internet research and technical community has always
endorsed, the value of open discussion, and the desirability of
fostering innovation and creativity, mean that we *need* the
independent submission channel. So please treat my following
comments in that positive context.

John has (at my request after seeing the -00 version)
moved the implications for BCP 92 (RFC 3932) into an appendix.
But I will take this first since it leads to my main concern:

>    3.  Some, although certainly not all, independent submissions are
>        extensively discussed and reviewed within the IETF community, or
>        the Internet community more generally, prior to submission for
>        publication.  Just as no statement should appear in a published
>        document that states or implies level of adoption, agreement, or
>        consensus that did not occur, no statement should appear that
>        states that significantly less review occurred than was actually
>        the case.  Similarly, no statement should appear that suggests
>        that the IETF community has a negative view of the document
>        unless such a view has been established via formal consensus-
>        determining mechanisms.

It may well be the case that an independent submission has been reviewed
by people who are active in the IETF. But that is not the same as it being
reviewed explicitly by a WG, a team such as the MIB Doctors, etc. Also,
BCP 92 intentionally limits IESG review (and by implication, any review
in the IETF requested by the IESG) to look at potential *interference*
with IETF work. So the IESG Note inserted in these documents really
can't take account of anything but the absence of IETF technical review.
To be precise, the form of words defined in BCP 92 is:

       The IETF disclaims any knowledge of the
       fitness of this RFC for any purpose and in particular notes that
       the decision to publish is not based on IETF review for such
       things as security, congestion control, or inappropriate
       interaction with deployed protocols.  The RFC Editor has chosen to
       publish this document at its discretion.  Readers of this RFC
       should exercise caution in evaluating its value for implementation
       and deployment.  See RFC 3932 for more information.

I don't believe that this is pejorative and I don't believe it needs
to be changed; I suspect John may disagree.

Assuming we get past this point, I now quote from the end of section
1.2:

>    That role is compromised if IETF-related entities are able
>    to block or deprecate such documents to a degree beyond that needed
>    to avoid difficulties with the standards process.  While the authors
>    and contributors to this document are firmly committed to IESG review
>    to identify conflicts with the standards process and suggestions that
>    would cause serious harm to the Internet, as outlined in RFC 3932 and
>    RFC 2026 [RFC2026], they believe that the IESG should deprecate, as
>    distinct from issuing general warnings about the lack of formal IETF
>    review, only those things about which there is IETF consensus about
>    harm.

Firstly, under BCP 92, the IESG can issue a Do Not Publish recommendation
to the RFC Editor only in three cases, defined as:

    3. The IESG thinks that publication is harmful to the IETF work done
       in WG <X> and recommends not publishing the document at this time.

    4. The IESG thinks that this document violates IETF procedures for
       <X> and should therefore not be published without IETF review and
       IESG approval.

    5. The IESG thinks that this document extends an IETF protocol in a
       way that requires IETF review and should therefore not be
       published without IETF review and IESG approval.

The latter two are recommendations for IETF review and therefore
consistent with draft-klensin-rfc-independent. The first one is not.

I object to John's text just quoted because it appears to call for
a requirement for establishing IETF consensus prior to the IESG
issuing a Do Not Publish recommendation of type 3. I believe this
waters down the IESG's authority, but more seriously it will create
work for the whole IETF in establishing such consensus. One point of
RFC 3932 was to reduce cycles spent on independent submissions.

In practice, the AD shepherding an independent submission through BCP 92
review will always consult the relevant WG chairs and/or expert reviewers,
but this is an efficient process since it does not involve a formal
consensus requirement.

I do not believe it is a wise use of resources to ask for "IETF consensus
about harm" and this is a change to RFC 3932 that I am against.
Putting a duty on the IESG to request expert input would be fine,
since that is the common-sense practice anyway.

> 4.5.  Review and Evaluation
...
>    While the reviews will generally be public, as discussed above,
>    reviewers are allowed to be anonymous at their request.

I know John has arguments for this, but it seems wrong to me. This
is not a peer-reviewed academic journal.

> 4.8.  Formal IESG Review

I object to this section being anything other than a reference
to BCP 92. I believe it is quite inappropriate to paraphrase
or otherwise duplicate approved process documents.

>    The RFC Editor or the author may request that the IAB review the
>    IESG's request to delay or not publish the document and for an
>    additional opinion.  Such a request will be made public via the RFC
>    Editor web site.  As with the IESG review itself, the IAB's opinion,
>    if any, will be advisory.  And, as with author requests for an IAB
>    technical review (see Section 4.7), the IAB is not obligated to
>    perform this type of review.

A fortiori, I object to this. It also seems pointless, since the
RFC Editor is assumed to have the right to ignore a Do Not Publish
request, in which case the IESG may request the insertion of an IESG
Note in the RFC.

> 5.  The Editorial Review Board
...
>    ...  However, to ensure the independence of the
>    independent submission process, the final decision to appoint (or not
>    appoint) Editorial Board members rests with the RFC Editor.

Then how is the Board made responsible to, and responsive to,
the community? Is this a job for life, and how can the community
express dissatisfaction? I find this very much at variance with
the style of community-based appointment we've adopted in the
IETF and elsewhere.

NIT:

> 4.  Review
> 
>    While this document is consistent with the broad outline of
>    independent submission and review as practiced over the years, it
>    specifies some new arrangements in RFC Editor processing that will
>    improve the balance between openness and independent decisons.
>    [RFC3932], specified its view of its role in the review process for
>    independent submissions.

The last sentence seems to be an editing victim.

     Brian (speaking for myself)

_______________________________________________
INDEPENDENT mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/independent

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 09 Jun 2006 08:21:40 -0400

Two things about the selection of responses strike me reading your note.
Firstly, I realized that I do not understand what we intended when we 
as a community adopted condition 3 above.  I presume we intended 
something reasonable, and suspect I thought I knew what was meant 
when I read it many years ago.  But I can not figure it out now.  How 
can a document be "harmful to the IETF work done in WG <X>."  I don't 
think this is just intended to catch documents which misrepresent 
existing work.
After all, it seems to me that a generally useful review comment, 
from anyone, is "this document mis-describes work X, Y, and Q.  It 
should be fixed before publication." (Or, to use Eric's words, it is 
factually incorrect.)

The other aspect which affects my response is that as you note later, 
point 3 does not inherently result in blocking publication.  Instead, 
it may result in blocking if the RFC Editor agrees (like any useful 
review) or it may result in an IESG note being added.   The question 
of what scope of notes to permit the IESG to add to independent 
submissions is a different question from what sort of blocking to 
permit the IESG.  Which question where you trying to raise?

Yours,
Joel M. Halpern

---

From: Craig Partridge <[email-masked]>
Date: Fri, 09 Jun 2006 09:13:07 -0400

>to be changed; I suspect John may disagree.

I believe it is highly perjorative.  Running from most to least important
point:

    * "Readers of this RFC should exercise caution in evaluating its
	value for implementation and deployment."

    That's a "this may not work statement." Furthermore, it suggests
    the IESG has reviewed the document in detail enough to express
    skepticism and it disagrees with the assessment (if the document
    describes a protocol) that the RFC Editor made to publish.

    * More generally, and consistent, with the previous point -- it
    suggests the IETF somehow views itself as the only entity capable
    of judging technical content -- else, why would the IETF need
    to insert a statement like this in the first place?

    * Indeed, why should the IETF say anything?  Having lived through
    much of the RFC = standard debate, I understand that one may want
    to say something (how's about "This document is not an IETF standard"?)
    but this text is certainly overkill.

>RFC 3932 was to reduce cycles spent on independent submissions.

I don't believe that's our intent (and, BTW, John's draft reflects
a larger concensus on the editorial side).

There are two issues here (and, as I recall, John's text is focused on
the first):

    * an IESG director (and that's usually what IESG review comes down
      to) should not be putting perjorative text (see above) into
      a document without IETF review.  That's distinct from saying
      "please hold this up as it may cause confusion."

    * how does an author appeal an IESG AD's delay on a document?  Again,
      most of us have been around the IESG process enough to see 
      abuses.  What's the solution?  Your note suggests the answer
      is the RFC Editor ignore the IESG -- that's an invitation for
      a food fight -- why not have a simple appeal process?

>the style of community-based appointment we've adopted in the

Yes, but mechanically it clearly has to work this way.  Consider,
the editor has to be able to fire any member of the editorial board
for non-performance.  Ultimately, anyone who has the authority to
fire, fundamentally determines who is appointed.  Q.E.D.

Craig

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 12 Jun 2006 15:04:44 +0200

Craig Partridge wrote:

<snip>

>       "please hold this up as it may cause confusion."

I'm not going to comment at this point on the 'pejorative paragraph'
question but I'd be glad to see other opinions on it.

>       a food fight -- why not have a simple appeal process?

Delay is one of the hardest things to appeal against, especially in
a volunteer community. So there's no easy answer here. But what I
meant to convey is that if the RFC Editor has the ultimate right
to override an IESG DNP request, I'm not sure what an appeal to the
IAB wins for the independent submission channel. So far, in my
experience, the difference between an appeal and a food fight is
hard to identify.

> fire, fundamentally determines who is appointed.  Q.E.D.

No, sorry, this doesn't satisfy me. Coupled with the right to make
anonymous reviews, there is a door open for potential misuse of
authority here. I really think we can do better, e.g. a recall
petition process as for IESG/IAB slots. (Needless to say, I am
not concerned with any of the present incumbents.)

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 12 Jun 2006 14:47:42 +0200

Joel M. Halpern wrote:
> Firstly, I realized that I do not understand what we intended when we as 
> a community adopted condition 3 above.  I presume we intended something 
> reasonable, and suspect I thought I knew what was meant when I read it 
> many years ago.  But I can not figure it out now.  How can a document be 
> "harmful to the IETF work done in WG <X>."  I don't think this is just 
> intended to catch documents which misrepresent existing work.
> After all, it seems to me that a generally useful review comment, from 
> anyone, is "this document mis-describes work X, Y, and Q.  It should be  
> fixed before publication." (Or, to use Eric's words, it is factually 
> incorrect.)

Let me try to construct an example that would qualify. Somebody
sends a draft to the RFC Editor that proposes to use an already
allocated IP option in a different way to what is being discussed
in an ongoing WG draft; furthermore the two different usages
would lead to direct ambiguity about what a host receiving such an
option should do with the packet. You can't blame the RFC Editor's
reviewer for not knowing this; s/he isn't omniscient. But it would
be a clear case 3.

> point 3 does not inherently result in blocking publication.  Instead, it 
> may result in blocking if the RFC Editor agrees (like any useful review) 
> or it may result in an IESG note being added.   The question of what 
> scope of notes to permit the IESG to add to independent submissions is a 
> different question from what sort of blocking to permit the IESG.  Which 
> question where you trying to raise?

Neither, to be honest: I am happy with RFC 3932 as it stands and I would
be sorry to see John's draft be inconsistent with it.

     Brian

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 Jun 2006 11:14:31 -0400

--On Monday, 12 June, 2006 14:47 +0200 Brian E Carpenter
<[email-masked]> wrote:

>...
>>>    3. The IESG thinks that publication is harmful to the
>>>    IETF work done in WG <X> and recommends not publishing
>>>       the document at this time.
>...
>> "harmful to the IETF work done in WG <X>."  I don't think
>> this is just  intended to catch documents which misrepresent
>> existing work. After all, it seems to me that a generally
>> useful review comment, from  anyone, is "this document
>> mis-describes work X, Y, and Q.  It should be   fixed before
>> publication." (Or, to use Eric's words, it is factually 
>> incorrect.)
 
> reviewer for not knowing this; s/he isn't omniscient. But it
> would be a clear case 3.

Brian,

I am glad you chose the term "construct an example", because
this sort of case, in reality, never happens.  The RFC Editor
and the Editorial Board, at least the current group of people,
have been exceptionally sensitive to this sort of issue.  The
general practice has been that, if even a hint of such a
situation occurs, the relevant AD is asked, informally, for an
opinion even before other issues are addressed that would lead
to a formal request for IESG review.  If the answer comes back
"of course this is an end run" or "yes, that would conflict with
an option that already exists", the author is told that either a
clearly non-conflicting draft is needed or the document must be
processed via the IESG, but the conflicting draft is dead as far
as the independent submission process is concerned.

I also note that there has historically been some coordination
between the RFC Editor with regard to any use of a potentially
allocated option and the IANA and IANA databases.  Again, the
identification of such a conflict would rapidly send the author
back to the drawing board.

So, at least in my experience, while the formal IESG review is a
useful additional safeguard, the odds of the IESG saying "DNP"
with that sort of explanation and the RFC Editor deciding to
publish with a disclaimer are close enough to zero to be
irrelevant.   

As I indicated in my earlier note this morning, the RFC Editor
has consistently, for 30-odd years, operated in the best
interests of the Internet community and this clearly would not
be.

So I think Joel (and I) are still missing a realistic example of
what this is about.

>> The other aspect which affects my response is that as you
>> note later,  point 3 does not inherently result in blocking
>> publication.  Instead, it  may result in blocking if the RFC
>> Editor agrees (like any useful review)  or it may result in
>> an IESG note being added.   The question of what  scope of
>> notes to permit the IESG to add to independent submissions is
>> a  different question from what sort of blocking to permit
>> the IESG.  Which  question where you trying to raise?
> 
> Neither, to be honest: I am happy with RFC 3932 as it stands
> and I would be sorry to see John's draft be inconsistent with
it.

>From my point of view, the only _major_ inconsistency between my
draft and RFC 3932 is that, under my draft, the IESG does not
get to demand a note that says or even implies, in part, "The
IETF has determined that this proposal may be bad news" unless
there has been an _IETF_ determination of that fact.  While I
think it would be silly, I would have little difficulty with a
note that said "the members of the IESG, acting on the basis of
what they believe would be IETF community consensus if the IETF
community had been asked, have determined that this proposal may
be bad news".  The point is that the latter tells the truth,
while the former constitutes a substitution of the IESG's
judgment for that of the community.

Put differently, the text in 3932 that reads "The IESG
thinks..." is exactly correct.  The objection is that it leads
to a statement about what the _IETF_ has concluded.

      john

---

From: Leslie Daigle <[email-masked]>
Date: Mon, 12 Jun 2006 18:36:15 -0400

Hmm...

For myself, I find I am less interested in the composition
of the editorial board as long as:

	1/ it's clear that the responsibility for any
	   decisions still lies with the RFC Editor (I'm
	   not suggesting there is anything in the current
	   draft that says otherwise)

	2/ there is adequate support for feedback on the
	   reviews themselves to make it clear (to the RFC
	   Editor, others) when the composition needs to
	   include new/different technical expertise.

	   This *may* be addressed enough already in the
	   document.

Beyond that -- there are any number of failure modes I could
suggest from experience and extrapolation.  But, if we're
sticking to writing the document of how things are *meant*
to work, I suggest it will be important to:

	. periodically publicly review that things are working the
	  way this document describes (RFC Editor @ IAB plenary
	  once a year -- not for #'s, but for substance?)

	. if not, or if other changes are required -- open
	  up this document for revision.

Leslie.

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 Jun 2006 19:14:34 -0400

> 	  once a year -- not for #'s, but for substance?)

Leslie,

Let me suggest an experiment that requires no process other than
that you and the RFC Editor agree it is useful to try.  And, in
the interest of relatively timely feedback, let me suggest it
for Montreal.

Suppose you ask the RFC Editor to produce a written report or
slide deck that parallels the reports produced in the last few
years.  Have that report posted to either the RFC Editor web
page or the IAB one in advance of the plenary (I have no
preference and assume you and Bob and Joyce and sort that out)
and announce its location via IETF-Announce.  If people have
questions, they can raise them on-list or offline during the
meeting.   Then you arrange an RFC Editor report/ discussion for
the plenary that focuses exclusively on the independent
submission process and how it is working, with as much of the
RFC Editor staff and Editorial Board as are available in the
front of the room.  Such a discussion, even 15 or 20 minutes
worth, could be helpful to both the RFC Editor and the community
in figuring out whether things are working the way we (or at
least the subset of "we" who show up at IETF plenaries or listen
remotely and can make comments via Jabber) want it to work.   If
it is useful, perhaps it would become an annual arrangement, as
you suggest.  If it is not, then the IAB and the RFC Editor get
to discuss whether some other approach would be more effective.

Just an idea...

    john

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 13 Jun 2006 07:37:52 -0700

> reviewer for not knowing this; s/he isn't omniscient. But it would
There's no need to construct a case.

See  this URL:

https://datatracker.ietf.org/public/rfc_editor_announcement.cgi?command=show_detail&accepted=no&ballot_id=1743

It concerns draft-kompella-ccc-02, and is dated March 23, 2006, with a 
fairly long "why" section.

Seems to be the only case so far that has actually used this variant of 
the boilerplate - out of a grand total of two DNP requests listed on the 
relevant page:

https://datatracker.ietf.org/public/rfc_editor_announcement.cgi?accepted=no

                  Harald

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 13 Jun 2006 10:55:27 -0400

That is actually a different case than the one I was asking about.
What is requested there is not "do not publish ever", but rather, "do 
not publish until the working group gets there stuff out the 
door."  As long as the working group is making progress (even if 
slow) that is a reasonable thing to ask.  There is a separate 
condition in Brian's list which covers that case.

Yours,
Joel

At 10:37 AM 6/13/2006, Harald Alvestrand wrote:
>>>Firstly, I realized that I do not understand what we intended when 
>>>we as a community adopted condition 3 above.  I presume we 
>>>intended something reasonable, and suspect I thought I knew what 
>>>was meant when I read it many years ago.  But I can not figure it 
>>>out now.  How can a document be "harmful to the IETF work done in 
>>>WG <X>."  I don't think this is just intended to catch documents 
>>>which misrepresent existing work.
>>>should be  fixed before publication." (Or, to use Eric's words, it 
>>>is factually incorrect.)
>>
>It concerns draft-kompella-ccc-02, and is dated March 23, 2006, with 
>a fairly long "why" section.
>
>Seems to be the only case so far that has actually used this variant 
>of the boilerplate - out of a grand total of two DNP requests listed 
>on the relevant page:
>

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 13 Jun 2006 23:55:15 -0700

Joel M. Halpern wrote:
> not publish until the working group gets there stuff out the door."  
> As long as the working group is making progress (even if slow) that is 
> a reasonable thing to ask.  There is a separate condition in Brian's 
> list which covers that case.
agreed - I included the words "at this time" in condition 3 after having 
the IESG go through exactly the same set of thoughts.

RFC 3932 doesn't give a way for the IESG to say that "this should NEVER 
be published" based on conflict with IETF work - and that was quite 
deliberate.
Also deliberate was the removal of the IESG's option to block a document 
based on percieved harm to the Internet. RFC 3932 does NOT give the IESG 
that option.

Apologies for not having read Brian's note before responding.
>>>> we as a community adopted condition 3 above.  I presume we intended 
>>>> when I read it many years ago.  But I can not figure it out now.  
>>>> How can a document be "harmful to the IETF work done in WG <X>."  I 
>>>> don't think this is just intended to catch documents which 
>>>> misrepresent existing work.