Subject: A Process Experiment in Normative Reference Handling

From: Brian E Carpenter <[email-masked]>
Date: Thu, 30 Mar 2006 14:46:39 +0200

It would be very interesting to have WG Chair feedback on
draft-klensin-norm-ref-00.txt

Good idea? Bad idea?

      Brian

Abstract

    The IETF and RFC Editor have a long-standing rule that a document at
    a given maturity level cannot be published until all documents it
    references as normative are at that maturity level or higher.  This
    rule has sometimes resulted in very long publication delays for
    documents and some claims that it was a major obstruction to
    advancing documents in maturity level.  The IETF agreed to a way to
    bypass this rule with RFC 3967.  This document proposes a one-year
    process experiment in which the "hold on normative reference" rule
    will be replaced by a "note downward normative reference and move on"
    approach.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 30 Mar 2006 19:17:37 +0200

Hi Brian,

> draft-klensin-norm-ref-00.txt

I think this may be a good idea, and worth the experiment.  I would like
Section 2.1 of draft-klensin-norm-ref-00 to be more clear about where the
additional note (bullet 2) is to be placed in the document, though, and
would suggest that this note should be appended to the document reference
in question, in the relevant References Section.

	Henrik

---

From: "James Kempf" <[email-masked]>
Date: Thu, 30 Mar 2006 08:24:06 -0800

So what would be the procedure once the normative references were published, 
update the RFC?

A counterindicative observation is that RFCs which are intended to be used 
by other SDOs are not likely to benefit much by this, since many SDOs will 
require that any normative references be published as well.

I think the real solution is to fix the RFC Editor process so long backlogs 
don't occur. The amount of copyediting that goes into publishing RFCs is 
quite trivial and it is somewhat mystifying to me why we have this problem 
(modulo the Auth 48 problem, which I think has been solved by the IESG).

            jak

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 30 Mar 2006 20:22:32 +0200

Hi James,

> (modulo the Auth 48 problem, which I think has been solved by the IESG).

I believe John's proposal is less directed towards the cases where a normative
reference of the same maturity level is in the pipeline, and more directed
to the case where no normative reference of the same maturity level is expected
any time soon, but normative references of a lesser maturity level exists
as published RFCs.  In this case, the size of the RFC editor backlog is
immaterial; the problem exists independent of that.

	Henrik

---

From: Peter Koch <[email-masked]>
Date: Thu, 30 Mar 2006 20:29:01 +0200

>    will be replaced by a "note downward normative reference and move on"

this appears more as an "experiment" than an experiment to me. I'd agree that
clearing the queue is a valuable goal, but i miss any measurement of
success or failure in terms of the actual experiment, i.e. what damage
would downrefs cause?

-Peter

---

From: "James Kempf" <[email-masked]>
Date: Thu, 30 Mar 2006 10:34:17 -0800

I don't think other SDOs would be concerned about the maturity level of an 
Internet Standard. Outside of the IETF, I believe Proposed Standard is 
considered sufficient as a reference for all practical purposes.

            jak

----- Original Message ----- 
From: "Henrik Levkowetz" <[email-masked]>
To: "James Kempf" <[email-masked]>
Cc: "Brian E Carpenter" <[email-masked]>; <[email-masked]>; 
<[email-masked]>
Sent: Thursday, March 30, 2006 10:22 AM
Subject: Re: A Process Experiment in Normative Reference Handling

---

From: Fred Baker <[email-masked]>
Date: Thu, 30 Mar 2006 17:03:21 -0800

personally, I would be in favor of publishing the document along with  
a reference to the internet draft for which there is a normative  
reference, and updating the reference in the document without  
changing the RFC number as those documents clear the queue. This is a  
major change to RFC Editor process (once published, it is considered  
chiseled into stone), but I would consider a change to the reference  
to be a change that does not affect the content of the document.

To my way of thinking, if that is not true (if a change to the  
referenced I-D would result in a material change to a document  
waiting for publication), then the document isn't ready for the IESG  
to put into the RFC Editor's queues at all.

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 30 Mar 2006 17:35:05 -0800

Hi -

> From: "Fred Baker" <[email-masked]>
> To: "Brian E Carpenter" <[email-masked]>
> Cc: <[email-masked]>; <[email-masked]>
> Sent: Thursday, March 30, 2006 5:03 PM

> to be a change that does not affect the content of the document.

I've always like the ISO approach where in the normative references
section there would be a statement like "At the time of publication,
the editions indicated were valid.  All standards are subject to revision,
and parties to agreements based on this International Standard are
encouraged to investigate the possibility of applying the most
recent editions of the standards listed below."  (The exact words
from the boilerplate have changed over the years.) 
 
> to put into the RFC Editor's queues at all.
...

Yup.

Randy

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 30 Mar 2006 22:14:08 -0500

>     will be replaced by a "note downward normative reference and move on"

It appears to me that John is proposing two different and only 
superficially related changes as part of the same experiment.  I believe 
that only one of these changes is necessary or appropriate, and the other 
should be shelved at least for now.

The first change is to the "no normative downreferences" rule, which says 
that a specification cannot advance along the standards track ahead of 
documents to which it has normative references.  The proposed change 
appears to be to eliminate this as a blanket rule, and instead require that 
any such downreferences be noted as part of the referencing document.  The 
intended effect appears to be that downreferences are evaluated on a 
case-by-case basis by the relevant working group and the community at large.

I support this change.  The Internet is a large system with a reasonably 
modular design.  We have many cases of protocols which build on other 
protocols by the use of well-defined interfaces, and components on opposite 
sides of such an interface can and should evolve and progress separately.

The second change is to eliminate the need for documents to be blocked in 
the RFC-Editor queue until all of their normative references have been 
published.  This would appear to have the effect of removing the 
requirement that published specifications include only stable references, 
by allowing an RFC to be published with an unstable normative reference 
such as an internet-draft filename.

This change, I oppose.  In my opinon, removing this requirement could have 
a serious negative affect on the usability of the resulting body of 
documents, as a reference to an internet-draft may become difficult or 
impossible to follow.  Such a draft might be renamed before being 
published, or even be split into a completely different set of documents. 
It is also likely to cause interoperability problems, as a result of 
different implementations being built against different versions of the 
referenced documents.

Finally, I think there's a meta-issue here, related to the lifetime of the 
experiment.  While the proposed experiment has limited lifetime, its 
effects will be more-or-less permanent.  I think for an experiment of this 
sort, it may be appropriate to limit the number of documents to which it 
can be applied before some analysis must be done to determine whether it is 
appropriate to proceed further.

-- Jeff

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 30 Mar 2006 20:00:07 -0800

I don't consider having to wait for a normative reference
to catch up to be a significant problem.  The document
gets published in due course.  I think not waiting, or
should I say, publishing without due course, would be
quite problematic.  The referenced I-D might never
appear (though I would hope the IESG would not approve
an I-D with a norm reference to not yet submitted I-D),
might morph significantly before being published (IMO,
the worse case), might never be published.

I do think we need to do more to encourage revision of
standard track documents to improve their quality.  I
personally don't see the downref rule as discouraging
revision work.  As I noted previously, the downref
rule encouraged me to take on revision work where I 
likely wouldn't have otherwise (I likely wouldn't have
worried about revising SASL specs if it wasn't for LDAP
spec's normative references to SASL specs).  However, I
could see that some might be discouraged by the rule.  I
have no problem with an experiment designed to determine
whether the rule discourages work to revise standard track
documents.

But I do think we need to limit the scope of the experiment,
namely by focusing on the what we can do to encourage
revision of standard track RFCs.

I believe it would be bad to allow an RFC to normatively
reference works in progress, or to allow standard track RFCs
to normatively references non-standard specifications. 

I suggest that we experiment with a relaxation of the
downref rule: allow standard track documents to normatively
reference other standard track documents regardless of
their standard track maturity.  That is, allow publication
of a DS that normatively references a PS.

- Kurt

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 30 Mar 2006 21:47:55 -0800

At 08:45 PM 3/30/2006, John C Klensin wrote:
>While the more general discussion of, e.g., what to do about I-Ds, is interesting and worth pursuing, all the document itself proposes is _exactly_ the above: references to published documents at earlier levels of the standards track and, at IESG/ community discretion, reference to approved (and hence firm, modulo only editorial work) I-Ds.   The latter are "editing in progress", not "works in progress" in the usual sense.

I don't read your I-D as being restrictive as my suggestion. 
You didn't qualify what you meant by "maturity", implying
that experiment allowed PS to normatively reference Informational
and Experimental RFCs as well as I-Ds.  My suggestion is not
to allow such references.

For clarity,  I can support an experiment that allows progression
(without the need for a variance) of a standard track RFC to DS or
FS that normatively references one or more standard track RFCs of
lessor standard track maturity.  I cannot support an experiment
that allows publication of a standard track RFC that normatively
references a non-standard track RFC or an I-D.

Kurt

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 31 Mar 2006 15:06:53 +0200

Kurt,

The phrase "maturity level" is from RFC 2026 and it means exactly
what you want. To grossly simplify John's proposal, it will
allow a DS to refer normatively to PS documents and a Standard to refer
normatively to DS and PS documents. We would need to decide whether
that's a problem or a solution.

     Brian

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 31 Mar 2006 05:30:17 -0800

At 05:06 AM 3/31/2006, Brian E Carpenter wrote:
>what you want.

I disagree.  RFC 2026 refers to all RFCs, including Informational
and Experimental RFCs, as having a maturity level.  RFC 2026
makes a clear distinction between standard track maturity levels
and non-standard track maturity levels.  

>To grossly simplify John's proposal, it will
>normatively to DS and PS documents.

John's I-D, as presently worded, appears to not to make any
distinction between RFCs on the standards track and those not
on the standards track. In fact, the I-D does not include the
phrase "standards track" (except in the title of one of the
references).  Hence the confusion the list.

John's proposal also allows RFCs to normatively reference
I-Ds.

>We would need to decide whether

If John's proposal would be revised to be clear as to
disallow a standard track RFC from referencing non-standard
track RFCs, as well as be changed to disallow referencing
I-Ds, I could support the revised proposal.  As it stands
now, I cannot.

>>I don't read your I-D as being restrictive as my suggestion. You didn't qualify what you meant by "maturity", implying

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Mar 2006 16:45:14 -0500

--On Thursday, March 30, 2006 20:22 +0200 Henrik Levkowetz 
<[email-masked]> wrote:

>> I think the real solution is to fix the RFC Editor process so
>> long backlogs  don't occur. The amount of copyediting that
>> goes into publishing RFCs is  quite trivial and it is
>> somewhat mystifying to me why we have this problem  (modulo
>> the Auth 48 problem, which I think has been solved by the
>> IESG).
>
> I believe John's proposal is less directed towards the cases
> where a normative reference of the same maturity level is in
> the pipeline, and more directed to the case where no normative
> reference of the same maturity level is expected any time
> soon, but normative references of a lesser maturity level
> exists as published RFCs.  In this case, the size of the RFC
> editor backlog is immaterial; the problem exists independent
> of that.

Yes, although the IESG, with appropriate community consensus, 
would certainly have the option to apply it in cases where 
waiting for the RFC Editor queue to clear was not considered 
appropriate.

--On Thursday, March 30, 2006 19:17 +0200 Henrik Levkowetz 
<[email-masked]> wrote:

> I think this may be a good idea, and worth the experiment.  I
> would like Section 2.1 of draft-klensin-norm-ref-00 to be more
> clear about where the additional note (bullet 2) is to be
> placed in the document, though, and would suggest that this
> note should be appended to the document reference in question,
> in the relevant References Section.

That was certainly what I intended.  If it isn't clear, better 
text would be welcome.

And, to respond to another comment, the RFC would not change 
when  the relevant documents were advanced in maturity level or 
published.  Language that said, approximately, "at the time this 
way published, that was not as mature", would remain true.  The 
question of how a reader would find a pointer to an upgraded 
version of the referenced specification, if one existed, is a 
separate one from the "downref" question -- one that the newtrk 
WG has had a number of proposals introduced to resolve.

      john

---

From: John C Klensin <[email-masked]>
Date: Fri, 31 Mar 2006 00:54:32 -0500

--On Thursday, March 30, 2006 22:14 -0500 Jeffrey Hutzelman 
<[email-masked]> wrote:

>
>
> On Thursday, March 30, 2006 02:46:39 PM +0200 Brian E
> Carpenter <[email-masked]> wrote:
>
>>     The IETF and RFC Editor have a long-standing rule that a
>>     document at a given maturity level cannot be published
>>     until all documents it references as normative are at
>>     that maturity level or higher.  This rule has sometimes
>>     resulted in very long publication delays for documents
>>     and some claims that it was a major obstruction to
>>     advancing documents in maturity level.  The IETF agreed
>>     to a way to bypass this rule with RFC 3967.  This
>>     document proposes a one-year process experiment in which
>>     the "hold on normative reference" rule will be replaced
>>     by a "note downward normative reference and move on"
>>     approach.
>
>
> superficially related changes as part of the same experiment.
> I believe that only one of these changes is necessary or
> appropriate, and the other should be shelved at least for now.
>
>
> The first change is to the "no normative downreferences" rule,
> which says that a specification cannot advance along the
> standards track ahead of documents to which it has normative
> references.  The proposed change appears to be to eliminate
> this as a blanket rule, and instead require that any such
> downreferences be noted as part of the referencing document.
> The intended effect appears to be that downreferences are
> evaluated on a case-by-case basis by the relevant working
> group and the community at large.
>
> I support this change.  The Internet is a large system with a
> reasonably modular design.  We have many cases of protocols
> which build on other protocols by the use of well-defined
> interfaces, and components on opposite sides of sFrom [email-masked] Fri Mar 31 16:53:56 2006
Date: Thu, 30 Mar 2006 16:45:14 -0500
From: John C Klensin <[email-masked]>
To: Henrik Levkowetz <[email-masked]>,
	James Kempf <[email-masked]>
Cc: [email-masked], Brian E Carpenter <[email-masked]>
Subject: Re: A Process Experiment in Normative Reference Handling

WG has had a nuuch an
> interface can and should evolve and progress separately.

> The second change is to eliminate the need for documents to be
> blocked in the RFC-Editor queue until all of their normative
> references have been published.  This would appear to have the
> effect of removing the requirement that published
> specifications include only stable references, by allowing an
> RFC to be published with an unstable normative reference such
> as an internet-draft filename.
>
> This change, I oppose.  In my opinon, removing this
> requirement could have a serious negative affect on the
> usability of the resulting body of documents, as a reference
> to an internet-draft may become difficult or impossible to
> follow.  Such a draft might be renamed before being published,
> or even be split into a completely different set of documents.
> It is also likely to cause interoperability problems, as a
> result of different implementations being built against
> different versions of the referenced documents.

Once again, I did not propose this, at least intentionally.  The 
_only_ case I anticipated permitting a reference to an I-D was 
after a Protocol Action had been been taken on that I-D.  To use 
the examples above, once such an action occurs, renaming, 
splitting, and protocol changes that would impact 
interoperability are impossible without violating other process 
constraints.   Should such references be permitted, they would 
make a strong[er] case for early assignment of permanent 
identifiers ("early" == "protocol action time") whether those 
identifiers were RFC numbers or some sort of independent 
document ID.   But, again, there is a huge difference between 
"RFC nnnn, publication pending" (post protocol action) and 
"draft-foo-bar-baz, work in progress" without a settled version 
number, etc.

> Finally, I think there's a meta-issue here, related to the
> lifetime of the experiment.  While the proposed experiment has
> limited lifetime, its effects will be more-or-less permanent.
> I think for an experiment of this sort, it may be appropriate
> to limit the number of documents to which it can be applied
> before some analysis must be done to determine whether it is

I'm  not convinced this is necessary, but have no objection.

     john

mber of proposals introduced to resolve.

      john

---

From: John C Klensin <[email-masked]>
Date: Fri, 31 Mar 2006 00:54:32 -0500

--On Thursday, March 30, 2006 22:14 -0500 Jeffrey Hutzelman 
<[email-masked]> wrote:

> interfaces, and components on opposite sides of such an

     john

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Mar 2006 23:45:38 -0500

Kurt,

While the more general discussion of, e.g., what to do about 
I-Ds, is interesting and worth pursuing, all the document itself 
proposes is _exactly_ the above: references to published 
documents at earlier levels of the standards track and, at IESG/ 
community discretion, reference to approved (and hence firm, 
modulo only editorial work) I-Ds.   The latter are "editing in 
progress", not "works in progress" in the usual sense.

    john

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 31 Mar 2006 14:23:06 -0800

Hi -

In the discussion of John Klensin's i-d, the distinction was made
between i-ds as "works-in-progress" and (post protocol action
announcement) "works-in-editing".  I think this is a valid distinction.

However, in another discussion happening at this very moment,
someone is claiming that the language in RFC 2026 doesn't
really support this distinction.  This seems like yet another case
of "do as we do, not as we say."

Randy

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 31 Mar 2006 16:17:24 -0800

>announcement) "works-in-editing".  I think this is a valid distinction.

While there certainly is a distinction in terms of process
state, as technical changes are often made post-IESG-approval,
the actual stability of these so-called "works-in-editing"
is (IMO) less than the process distinction or this term implies.

Kurt

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 31 Mar 2006 16:59:15 -0800

Few random comments ...

I like this idea but I don't see how you can do it as an experiment. At the
end of the year, if we decided not to continue with the experiment, would
all the documents go back to their previous status had we not run the
experiment? It will just cause a rush to publish stuff that is not ready to
slide it in under the 1 year rule. What would happen after the one year to a
bis of document that how downward refs? I like the idea of this draft but it
seems like IETF has to commit to it or not - it's going to be very hard to
do for one year then stop. If you do keep it as an experiment, I think you
need more about what happens after the one year.

In section 2.1, having some general (not necessarily binding) guidance of
when downward refs were acceptable would be good. I'm not sure what to put
but I think the right text here would help people be conformable with the
proposal. 

Reading the list traffic, I see that you did not intend this to allow
normative references to I-Ds that are not approved. It might be lost on some
people that this does expand the current rules to allow an RFC to normative
reference an I-D that is approved but not yet published. I assume this would
mean we would allocate the RFC numbers on approval of an I-D instead of on
publishing so that the text or referencing RFC did not have to be updated.
I'm OK with this but personally I would be very against normative references
to I-Ds that are not yet approved for reasons I have put on this list
before. 

Making it crystal clear what sort of downward refs were allowed under this
would be good.

Thanks for doing this - it seems like something that could break a huge log
jam. 

Cullen