Subject: Re: I-D ACTION:draft-klensin-norm-ref-01.txt

From: "C. M. Heard" <[email-masked]>
Date: Wed, 5 Apr 2006 08:03:02 -0700 (PDT)

This draft proposes the following:

   o  A note is included in the reference text that indicates that the
      reference is to a document of a lower maturity level, that some
      caution should be used since it may be less stable than the
      document from which it is being referenced, and, optionally,
      explaining why the downward reference is appropriate.

I would like to suggest a simpler alternative:  require that the downref
be noted at last call time (both WG last call and IETF last call), not
in the document itself.  This is in keeping with current practice of
not discussing standardization status of a document in the document
itself.  Moreover, such notes can become obsolete, since it is always
possible that a downref target will eventually be advanced.

Mike Heard

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 05 Apr 2006 18:42:39 +0200

Hi Mike,

> possible that a downref target will eventually be advanced.

I disagree, mainly because I think that it is more valuable to have the
warning clearly visible in the document.  If it's only noted during last
call we will have no plain-in-your-face document which shows future
readers the state of things.  If the referenced document is advanced
without publishing a new version, there may be a moment of surprise when
the reader notes that the document is now more advanced than when the
reference was made, but that's a better trade-off, I think.

Maybe it would help if the note in the reference explicitly mentioned
that the state of the referenced document was such-and-such "at the time
of writing", or "at date so-and-so"?

	Henrik

---

From: Thomas Narten <[email-masked]>
Date: Wed, 05 Apr 2006 14:13:05 -0400

> possible that a downref target will eventually be advanced.

Without necessarily taking a position on either of these, I'll note
that putting it only in the LC effectively means that readers of the
(RFC) document will never realize there is a downref, when maybe they
should.

Thomas

---

From: Eric Rosen <[email-masked]>
Date: Wed, 05 Apr 2006 15:18:12 -0400

If we really want to be able to progress documents that have downrefs, there
is a  very simple  solution, viz., simply  eliminate the rule  that prevents
this. Period. 

Don't replace it with some other rule that continues to maintain the fiction
that  a  downref is  a  deplorable circumstance  which  needs  to be  noted,
discussed, last called, whined about, justified, or processed in any way.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 5 Apr 2006 21:42:26 +0300 (EEST)

> possible that a downref target will eventually be advanced.

While the maturity level of a standards track document might change, 
the "maturity" of an Informational or Experimental RFC doesn't get 
upgraded. (It can go even lower to "historic" though, but I'm not sure 
if our current downref rules take a stance on normative references to 
Historic or Obsoleted RFCs.)

I think a note in the document is appropriate especially for these 
categories, and preferably also for standards track.

Btw. the document doesn't mention Experimental RFCs at all.  Is this 
an intentional omission, i.e., downreffing those is out of scope of 
this experiment?

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Bob Hinden <[email-masked]>
Date: Wed, 5 Apr 2006 13:19:37 -0700

> discussed, last called, whined about, justified, or processed in  

I think Eric's point is right on the mark.  We should just eliminate  
the rule.

We don't have a three step standards process anymore.  It's silly to  
have a rule that doesn't let a document advance just because one of  
the document it depends on isn't at the same level.  In many cases  
the working group that produced it is probably disbanded and no one  
has very much interest in advancing it.

Also, when moving to Draft standard or Standard, the important thing  
is that there are multiple interoperable implementations.  That is  
the important test for stability of the specifications.  If people  
can build multiple interoperable implementations, then the  
specification and references are adequate.

Bob

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 05 Apr 2006 21:15:29 -0400

On Wednesday, April 05, 2006 01:19:37 PM -0700 Bob Hinden 
<[email-masked]> wrote:

> I think Eric's point is right on the mark.  We should just eliminate  the
> rule.
>
> We don't have a three step standards process anymore.

On paper, we do.  It strikes me that the fact that many documents never 
progress as far as draft means we aren't doing the task of examining 
implementation and deployment experience and updating specifications to 
reflect that.  Which means that instead of standards that represent "rough 
consensus and running code", we have pseudo-standards that represent "rough 
consensus about what we hoped the running code would eventually look like". 
Perhaps we should be doing something about this, as I think we'll have a 
bit of trouble fitting the latter phrase on a cake 20 years from now.

I agree that if we don't have a three-step standards process, then having a 
rule about when things can progress from one step to the next is silly. 
But then again, so is spending any time whatsoever on making it easier to 
progress things through the three-step process we no longer have.

Maybe the proposed experiment will turn out to be pointless.  Or maybe it 
will demonstrate that we actually _do_ have a three-step standards process, 
but you can't tell because a few slackers are holding up everyone else's 
documents.  I think only time will tell.

-- Jeff

---

From: Bob Hinden <[email-masked]>
Date: Thu, 6 Apr 2006 11:00:12 -0700

Larry,

On Apr 5, 2006, at 5:39 PM, ext Larry Masinter wrote:

> Yes. Often the reference is just to a small part of the target.
> When asking about 'multiple interoperable implementations of each
> feature', the feature list should include the features that
> are specified in the normative references, if those references
> aren't otherwise at the same level. But then it wouldn't matter
> if the references are Proposed or Draft or Experimental or not
> even IETF documents at all (as long as they were stable references.)

I agree.  The real test is in the interoperability testing.

To draw a comparison, we use the interoperability testing to  
effectively validate if IPR on a document is really accessible.   
Seems to me if we can do this for IPR, then the same test also work  
for references.  Or in other words, worrying a lot about normative  
references and not worrying about IPR (a real problem) makes little  
sense.

Bob

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 07 Apr 2006 10:54:50 +0200

John C Klensin wrote:
> 
> --On Wednesday, 05 April, 2006 21:42 +0300 Pekka Savola
> <[email-masked]> wrote:
> 
> 
>>Btw. the document doesn't mention Experimental RFCs at all.
>>Is this an intentional omission, i.e., downreffing those is
>>out of scope of this experiment?
> 
> 
> It was intentional.  It may or may not be correct, so input is
> welcome.

As far as I understand the current rules, Experimentals can refer
normatively to any level of document (except work in progress),
so I'm not sure what change we could make.

     Brian

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 5 Apr 2006 14:19:24 -0700 (PDT)

> discussed, last called, whined about, justified, or processed in any way.

That sounds good to me.  In fact I like it better than my own suggestion.

//cmh

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Apr 2006 18:19:41 -0400

--On Wednesday, 05 April, 2006 15:18 -0400 Eric Rosen
<[email-masked]> wrote:

> If we really want to be able to progress documents that have
> downrefs, there is a  very simple  solution, viz., simply
> eliminate the rule  that prevents this. Period. 
> 
> Don't replace it with some other rule that continues to
> maintain the fiction that  a  downref is  a  deplorable
> circumstance  which  needs  to be  noted, discussed, last
> called, whined about, justified, or processed in any way.

Eric (and others),

Over the years, the principal IETF claims to fame and authority
relative to many other standards bodies are that we recognize
the importance of reality proofs by implementation and that we
are very clear about how stable, implementable, and mature we
consider particular specification to be.   One can believe that
is not longer important in this day and age.  One can believe
that not trying to make those distinctions has never hurt
others.  Or one can believe that we have irretrievably fouled
out own nests by failure to go to the effort to take most
specifications beyond Proposed Standard.

One can reasonably argue that our traditional distinctions
between those things we know, from demonstration and experience,
can be implemented and those for which we have merely written a
specification are no longer useful.  One can also argue that the
ways we traditional use to identify various levels of confidence
in a specification are no longer optimal, or that the approval
thresholds we have created are inappropriate, and should be
changed.  There are proposals in support of many of these views
in the I-D pile; I've even written several of them.

For better or worse, the traditional "downref" prohibition is
very much a part of the model of making distinctions among
levels and quality of proof-by-implementation.  If one is going
to build a specification that relies on another, presumably less
stable or proven, specification, then that requires an
explanation at least.  The analogy to building code that depends
on something that is not known to work is fairly obvious, at
least to me.

So, as a matter of efficiency in discussion, if nothing else,
let's have whatever discussions are appropriate about
eliminating the "implementation maturity" distinctions from the
standards track, or of tuning the ways we talk about or
determine maturity levels, in that specific context and let's
have them explicitly.  Proposing to weaken those rules by
eliminating the downref distinctions entirely does not really
move us forward to fewer distinctions, nor does it make
references work better in the current context.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Apr 2006 19:40:13 -0400

john

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Apr 2006 19:45:35 -0400

--On Wednesday, 05 April, 2006 14:13 -0400 Thomas Narten
<[email-masked]> wrote:

"C. M. Heard" <[email-masked]> wrote...

>>  This is in keeping with current practice of
>> not discussing standardization status of a document in the
>> document itself.

But, for downrefs (modulo the fairly stringent procedure in RFC
3967), normative downrefs cannot be used at all, so there is no
"current practice" for using them: the presence of a reference
implies equivalent or greater maturity level.

>> Moreover, such notes can become obsolete,
>> since it is always possible that a downref target will
>> eventually be advanced.

Which is why the proposed text suggested "at the time of this
writing" or words to that general effect.  It is perfectly
reasonable for the user who cares to check various indices,
etc., to determine whether the warning still applies.  Of
course, if no warning appears, then there is no warning, which
has been advocated elsewhere.

> Without necessarily taking a position on either of these, I'll
> note that putting it only in the LC effectively means that
> readers of the (RFC) document will never realize there is a
> downref, when maybe they should.

Yes, exactly.  See my earlier note to Eric Rosen.

     john

---

From: Larry Masinter <[email-masked]>
Date: Wed, 05 Apr 2006 17:39:17 -0700

Larry

---

From: Eric Rosen <[email-masked]>
Date: Wed, 05 Apr 2006 23:14:06 -0400

> Over  the  years  ...  [followed  by  several  paragraphs  discussing  the
> fundamental principles of the IETF  standards process and the way in which
> the downref rule is an essential linchpin of those principles ]

This is  an excellent example of why  we never make any  progress on process
issues.   Even the simplest  suggestion for  an experiment  gets immediately
bogged  down in  heavy philosophical  discussions of  fundamental principles
that, quite frankly,  don't seem to matter to anybody  except the people who
like to spend  time engineering the process.  And the  result, of course, is
that folks start to discuss various ways of making the process more and more
complicated, like engineers suffering from second system syndrome.

The downref rule is admirable  in its simplicity, but the working hypothesis
for the experiment is  that it does more harm than good.   I find it hard to
believe that  making the rule more  complicated ("you can have  a downref if
you fill out this form in triplicate ...") is the solution.

If we  are serious  about doing process  experiments, what  other experiment
would we do  than to eliminate the rule entirely,  and then evaluate whether
the elimination of the rule has been a  net gain or a net loss?  If it turns
out to be a  net loss, we can then discuss whether  some form of the downref
rule  needs to  be reinstated,  and at  least we'll  have  some demonstrated
problems to focus on.

We're supposed to be looking for what  works, not for what seems a priori to
accord with some set of philosophical principles.

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Apr 2006 23:53:36 -0400

--On Wednesday, 05 April, 2006 23:14 -0400 Eric Rosen
<[email-masked]> wrote:

> 
>> Over  the  years  ...  [followed  by  several  paragraphs
>> discussing  the fundamental principles of the IETF  standards
>> process and the way in which the downref rule is an essential
>> linchpin of those principles ]
> 
> This is  an excellent example of why  we never make any
> progress on process issues.   Even the simplest  suggestion
> for  an experiment  gets immediately bogged  down in  heavy
> philosophical  discussions of  fundamental principles that,
> quite frankly,  don't seem to matter to anybody  except the
> people who like to spend  time engineering the process.  And
> the  result, of course, is that folks start to discuss various
> ways of making the process more and more complicated, like
> engineers suffering from second system syndrome.

Eric,

I would have said the same thing about this discussion.  But we
seem to be citing the same principle and drawing opposite
conclusions about what to do about it.

> The downref rule is admirable  in its simplicity, but the
> working hypothesis for the experiment is  that it does more
> harm than good.   I find it hard to believe that  making the
> rule more  complicated ("you can have  a downref if you fill
> out this form in triplicate ...") is the solution.

And here I was, thinking that what I wrote was a simplification.
The rule prior to RFC 3967 was:

	(i) Figure out that you are making a normative reference
	to a document at a lower level.

	(ii) Last Call the document, knowing full well that, if
	it is approved, you will...
	
	(iii) Wait, or have the RFC Editor wait for you, until
	that condition clears.

Summary: No downrefs permitted

The rule imposed in RFC 3967 was, more or less:

	(i) Figure out that you are making a normative reference
	to a document at a lower level.
	
	(ii) Figure out if it is important enough to get the
	rules waived.  If not, wait and loop on (i).   If so, 
	
	(iii) Generate a last call on the matter of doing the
	downward reference and, if it is successful
	
	(iv) Proceed as if there were no downref.

Summary: No downrefs is still the default, but there is a
moderately complicated workaround for particularly problematic
cases.

Now, the proposed experiment is

	(i) Figure out that your are making a normative
	reference to a document at a lower level.
	
	(ii) Insert one sentence in the proposed RFC that notes
	the fact of the downref (not a complex form, in
	triplicate, or any other requirement I would categorize
	as burdensome)
	
	(iii) That sentence gets Last Called as part of the
	document, along with all of the other sentences in it.
	There is no extra procedure and, based on experience, we
	can assume that there will be no objections to the
	downref except in the most serious of conditions.

Summary: The "no downrefs" rule essentially disappears, replaced
by the bare minimum needed to preserve a distinction between
specifications that have been implemented and tested and those
that have not.  The special procedure of RFC 3697 remains
available, but probably no one uses it.

> If we  are serious  about doing process  experiments, what
> other experiment would we do  than to eliminate the rule
> entirely,  and then evaluate whether the elimination of the
> rule has been a  net gain or a net loss?  If it turns out to
> be a  net loss, we can then discuss whether  some form of the
> downref rule  needs to  be reinstated,  and at  least we'll
> have  some demonstrated problems to focus on.

Because (1) it would take years to figure out whether
eliminating the rule, and a good chunk of the IETF's norms for
distinguishing between what we are sure of and what is merely
design speculation, had caused harm or not and (2) while a case
can be made that the "note the downrefs" model that the I-D
proposes is relatively safe, there is even less of a path
backward from eliminating the rule entirely should we decide it
is a failure.  And because, at least in my view, complete
elimination of the downrefs is almost equivalent to complete
elimination of a multistage/ multilevel standards process and I
don't think there is any community consensus for doing that.

So, from my point of view, applying exactly the same principles
you are arguing for above, I listened to people complaining
about the negative impact the downref rule had on efficient
publication of standards.  I watched 3967 adopted and noticed no
drop in the number of complaints.  So I wrote the most
conservative proposal I could think of that effectively
eliminated any ability of the downref rule to cause delays if
the material being referenced were actually stable.  

If, after some relatively short period of time, we conclude that
the disclaimers are not useful but the relaxed rules are, then
dropping the disclaimers themselves is obviously easy.

The reactions haven't been "won't work" or "bad idea" or "we
like things just as they are".  Any of those, from my point of
view, would have been helpful: in many cases, this one included,
I generate process I-Ds less because I am convinced that they
are The Right Thing, but because I don't find it productive for
people to have extended mailing list discussions around
proposals that are too vague for anyone to be sure they are
talking about the same thing.  From that perspective, if the
consensus after I post a document like this is a clear "don't
want to do that", then the document is a complete success.  But
what we have seen instead, at least in my interpretation, has
mostly been "you need more rules because you can't trust the
judgment of the IESG and the community to do the right thing in
the edge cases" and many variations on "lets go discuss
fine-tuning the alternatives rather than doing anything".   We
agree that is a serious problem; we just disagree who is
proposing the fine-tuning and blocking progress.

> We're supposed to be looking for what  works, not for what
> seems a priori to accord with some set of philosophical
> principles.

And I think that is exactly what I'm proposing.  The question is
whether you are convinced enough that "references to anything"
and "no differentiation about degrees of confidence in
standards' are enough certain to work that we should go down
those paths with no way of going back.   We have some peer
standards bodies who did, and it nearly killed them in several
areas.  So, while you might categorize that as "looking for what
works", I would characterize it as "rushing madly ahead, eyes
closed, and with prior knowledge that there are cliffs in the
general direction we are running".

    john

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 7 Apr 2006 06:25:16 -0500

Hi, Brian,

I understood Pekka's comment to be about downrefs TO experimentals, although 
I may not have understood correctly (and apologize if I am confused).

(And nobody asked me, but) This is Yet Again about looking at references 
based on the class of document being referred to, and not looking at the 
document being referred to.

To be blunt, I'm not convinced that

- a reference to RFC 3742, "Limited Slow-Start for TCP with Large Congestion 
Windows", a Really Dangerous Experimental RFC that was simulated to death 
before publication, is a cause for concern, while

- a reference to STD 7, "TRANSMISSION CONTROL PROTOCOL", a Widely Deployed 
Standard RFC that contains no congestion avoidance whatsoever (because all 
this stuff was added in RFCs that are still Proposed Standards), is not a 
cause for concern.

So, I think Pekka's question, as I understand it, is an interesting 
question, if our standards levels are supposed to actually mean anything.

And if I understood Brian Rosen's point ("read the stinking references and 
SEE if downref looks like a problem"), it seems at least as reasonable as 
procedures based on standard level.

Thanks,

Spencer

From: "Brian E Carpenter" <[email-masked]>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 07 Apr 2006 14:45:32 +0200

Spencer Dawkins wrote:
> Hi, Brian,
> 
> I understood Pekka's comment to be about downrefs TO experimentals, 
> although I may not have understood correctly (and apologize if I am 
> confused).

You understood correctly; I was confused.

> - a reference to RFC 3742, "Limited Slow-Start for TCP with Large 
> Congestion Windows", a Really Dangerous Experimental RFC that was 
> simulated to death before publication, is a cause for concern, while
> 
> - a reference to STD 7, "TRANSMISSION CONTROL PROTOCOL", a Widely 
> Deployed Standard RFC that contains no congestion avoidance whatsoever 
> (because all this stuff was added in RFCs that are still Proposed 
> Standards), is not a cause for concern.

Sure. But there you are on another issue. A reference to STD 7
should clearly not be to RFC 793 alone. The notion that the TCP
standard consists only of RFC 793 is a strange fiction.

> question, if our standards levels are supposed to actually mean anything.

Nevertheless, if something has been published as Experimental, that is
a health warning right there. My personal opinion is that a real variance
is needed there. In the context of John's proposed experiment, this seems
a bit of a stretch.

> And if I understood Brian Rosen's point ("read the stinking references 
> and SEE if downref looks like a problem"), it seems at least as 
> reasonable as procedures based on standard level.

 From a purely personal view, I tend to agree, but again that isn't the
context, which is a proposed experiment that doesn't require us to
change any basic precepts.

     Brian

> .
> newtrk resources:_____________________________________________________
> web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
> mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html
> 
>