Subject: RE: [newtrk] last call on newtrk draft charter

From: "Christian Huitema" <[email-masked]>
Date: Sun, 8 Feb 2004 11:01:52 -0800

> > The concept of a WGS disturbs me, because it seems like we're
looking
> for
> > a way to publish the equivalent of a PS document that hasn't
necessarily
> > been through any type of cross-area review...

Conversely, I like the idea of a WGS or some equivalent precisely
because we can get something published without being taken hostage by
the umpteen blocking points that have been inserted in the process.

Another advantage is to act as a forcing function. In practice, once a
document is published, it can only be superseded by a "better" document,
i.e. one that reaches a wider consensus in the working group. This
prevents the common delaying tactics that leads to most of the delays,
and forces the WG to address perceived issues in a timely manner.

> > Do you (the newtrk WG, not just you John) really believe that WGs
should
> be
> > able to publish stable, long-lived specifications without the need
to
> pass
> > through any cross-area review and approval process?

Yes to long-lived, as in likely to remain available for an unlimited
period of time. No to stable: the specifications can be revised, etc.

> The IETF likes that people implement things before they become a PS.
> Interop
> events have been part of the IETF culture for a long time.  Various
IETF-
> ers
> have made fun of other SDOs that don't have any notion of running
code.
> However,
> we are now in a situation where many companies don't want to invest
the
> cost
> to implement things when there is no notion that the standard is
stable
> or has the chance of becoming a PS within a reasonable timeframe.  

I have observed both sides of this issue. On one hand, you are not going
to revise an existing product just because someone somewhere has written
a draft that may or may not become a standard some day. On the other
hand, if you really want to ship a specific function, you are going to
ship it whether or not the IETF converges on a specification. I am
concerned that the latter is happening more and more often.

> I think
> that the WGS has support precisely because it is damn hard to justify
> the expense of implementing something to cost-conscious management if
> you are not reasonably sure that a protocol is realively stable.  I
have
> noticed a big downturn in interop events with new IETF protocols, and
> I think that this might be one of the reasons why.

At a minimum, the WGS enables meaningful interoperability tests.
Otherwise, you get someone who implemented draft-11 testing against
someone who implemented draft-13, and disagreeing on the proper
resolution of issue X that happens to be addressed in draft-15... 

Part of the problem is that there is very little change control for
internet drafts. It is essentially up to the editor. The advantage of a
stable specification (WGS or other) would precisely be to have a change
control mechanism.

> Alternatively, we could keep going as we are going, taking 3 or more
years
> to produce a protocol and wait until after we have a PS to expect
folks
> to implement it.

That is not what is happening. The IETF takes 3 years to define a PS,
and in the mean time the industry implements something else, or moves
on.

-- Christian Huitema

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Keith Moore <[email-masked]>
Date: Sun, 8 Feb 2004 15:30:19 -0500

> the umpteen blocking points that have been inserted in the process.

as long as the net can be taken hostage by vendors deploying a protocol that
does harm, I think IETF needs to be able to take specifications hostage until
those specifications are fixed.  it won't stop vendors from deploying garbage
but it will at least create some doubt in the public mind that such garbage
is stable and well-designed.

naturally we'd like to find a better mechanism for producing good specs
in finite time, but bypassing external review doesn't sound like the ticket.

Keith
.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 8 Feb 2004 14:50:54 -0600

Not picking on Keith, only starting at the bottom of the thread that
I've seen so far.

There is one school of thought that says we could get BETTER external
review, inside and outside the IETF, if we had a way to say "this is
what the working group thinks, so far".

At this time, the only way we have to do this is publish a PS. Is the
only way we have working well for you?

At my "sponsoring organization", it would be easier to ask other
engineers to look at a WGS than a random Internet Draft.

Your fellow engineers may have time on their hands. Mine do not.

And it is really unfortunate that we are still trying to logic this
out six months after it was proposed, instead of attempting an
experiment.

Spencer

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Christian Huitema" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>;
<[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Sunday, February 08, 2004 2:30 PM
Subject: Re: [newtrk] last call on newtrk draft charter

> newtrk
resources:_____________________________________________________
> mhonarc archive:
http://darkwing.uoregon.edu/~llynch/newtrk/index.html

.

---

From: Pekka Savola <[email-masked]>
Date: Mon, 9 Feb 2004 07:57:06 +0200 (EET)

On Sun, 8 Feb 2004, Spencer Dawkins wrote:
[...]
> - WG says draft-ietf-woozle-ipOverHamsterWheels-09.txt is looking good
> - WG chair requests reviews from Area Review Team
> - ART sends back reviews that say "fix these three things"
> - WG fixes two things and says the third thing is really OK
> - WG chair sends -10.txt, the reviews, and an explanation of why the
> third thing is not a problem to the AD
> - the AD thinks -10.txt makes sense (given the reviews and the
> explanation of why the third thing is not a problem) and says "include
> the words 'This Internet Draft is issued as a Woozle Working Group
> Snapshot' in -11.txt.

Well.  If the WG already had the document reviewed, I would argue it's 
only a couple of relatively short steps to go to get to PS (or the 
like): an optional IETF Last Call and IESG review.  IESG review 
usually results in a required revision of the memo, but if the 
document has seen external review before, it's unlikely that these 
would be show-stopper problems.  

That is, my very strong perception is that the majority of time in the
process is spent by a) the editor of the document (not) revising the
document, and b) the WG getting stuck in an issue, unable to make a
decision how to move forward.  Compred to that, getting IESG to review
a document is irrelevant.

So, it seems like getting from a WGS state to an IESG approved stage 
shouldn't take more than a month, if the editor, WG chair and maybe 
the WG is active.

Thus, would a WGS state be useful at all (at least in a scenario like 
this)?

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

.

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 08 Feb 2004 21:39:57 -0500

At 07:57 PM 2/8/2004 -0500, Keith Moore wrote:
>It's completely broken for a WG to declare consensus on anything resembling
>a complete spec that hasn't enjoyed significant external review.  What this
>means is that the WG has become too committed to details before it has had
>a chance to get clue input from outside the WG.  That's a big part of what
>is wrong with our current process, and draft-dawkins-pstmt-twostage would
>only make it worse.

This sums up my concern pretty well.  BTW, I don't believe that "external
review" necessarily means IESG review.  This might be a good use for an
early/community review process that comes out of the ICAR effort, for example.

>Which, come to think of it, is a good example of why "newtrk" needs to take
>a slightly broader look at the process rather than just tweaking the standards
>track.

I think that we can talk about our standards track and the criteria for
entry at each stage, without necessarily having the same WG define the
processes to support achieving/measuring those criteria.

To my way of thinking, all document stages need to have some criteria
for entry, and some way that a decision is made about whether those
criteria have been met.  For instance, we could include some level of
cross-area review in the requirements for a WGS-type document, and
require the WG chair (or responsible AD?) to sign-off that the appropriate
reviews have been completed.  Or, we could require WG consensus (as
determined by the chair) that the appropriate reviews have been completed.

I do wonder, though...  If a WG can publish a WGS-type document based
on WG consensus (as determined by the chair), does that have any
implication for situations where the WG chair is also the document
editor?  Can we reach a point where only one person is involved in
judging whether a particular document is ready for WGS status?

Margaret

.

---

From: Keith Moore <[email-masked]>
Date: Sun, 8 Feb 2004 22:17:18 -0500

On Sun, 08 Feb 2004 21:39:57 -0500

> early/community review process that comes out of the ICAR effort, for

I'd like to see some of each, but I certainly don't think the IESG should
get in the loop at each step.  More likely, the AD for the group would be
in the loop and the IESG would only review progress say once a year.
 
> >a slightly broader look at the process rather than just tweaking the
> >standards track.
> 

I'm sure we can talk about those things separately.  I seriously doubt
that it's productive to do so.  If IESG insists on doing things this way
then the best result we can hope for is for the two groups to act as if
they're one group (ignoring the group boundary) or for one group to do the
work of both and for the other group to be dormant. 

> judging whether a particular document is ready for WGS status?

The notion of WGS as in "statement of WG consensus" is so fundamentally
flawed in so many ways that I think it's unfixable.

Keith
.

---

From: Keith Moore <[email-masked]>
Date: Sun, 8 Feb 2004 19:57:01 -0500

> engineers to look at a WGS than a random Internet Draft.

I'm in agreement with this much.  However, I have a lot of objections to
draft-dawkins-pstmt-twostage.

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 8 Feb 2004 20:50:38 -0600

From: "Keith Moore" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Sunday, February 08, 2004 6:57 PM
Subject: Re: Using WGS to bypass external review (was: Re: [newtrk]
last call on newtrk draft charter)

> draft-dawkins-pstmt-twostage.

I haven't requested an IETF last call yet, so I must think
draft-dawkins-pstmt-twostage can be improved :-}

> is wrong with our current process, and draft-dawkins-pstmt-twostage
would

OK, can we define "external review" precisely enough to say "if you
show up with evidence of working group consensus and external review,
that's good enough for a WGS"? If we can, I agree with this
suggestion.

FWIW, I am not opposed to reviews, only slow reviews.

> a slightly broader look at the process rather than just tweaking the
standards
> track.

My opinion is, if we can specify criteria for declaring WGS that do
not require the IESG to do anything except write "this is a horrible
idea, and here's why, in one hundred words or less" when the WGS, and
WG, is in the weeds, that would be a good thing. In
draft-dawkins-pstmt-twostage, IESG health warnings are explicitly
permitted, not discouraged.

I think I hear you suggesting that I amend this to say that the IESG
doesn't have to do anything, unless the WGS, the WG, AND the external
reviewers are all in the weeds.  _Slightly_ broader.

So, I agree here, too. If the WG and the external reviewers head into
the weeds all the time, we have bigger problems than IESG overload.

If the IESG has to actually read WGS and agree that publishing them is
OK, we already have WGSs, we just call them Proposed Standards.

Spencer

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 8 Feb 2004 21:30:28 -0600

From: "Margaret Wasserman" <[email-masked]>

> early/community review process that comes out of the ICAR effort,
for example.

This would be a step forward.

> determined by the chair) that the appropriate reviews have been
completed.

Either of these would be a step forward. They are not mutually
exclusive (see below).

> judging whether a particular document is ready for WGS status?

draft-dawkins-pstmt-twostage was big on concepts and lite on process,
so I'm thinking out loud here. With that warning ("you are entering a
twisty maze of passages, all alike")...

For this scenario (I may have misunderstood Margaret, but I was trying
to use both of her suggestions), one person would have to be the
editor, WG chair, and shepherding AD to lose checks and balances
(short of nomcom/AD recall).

And it would be lovely if there was a web page registering WGSs, that
you didn't get on without a note copying your AD, so just sending in a
bogus WGS didn't work very well...

Does this help?

If so, all we need to do is approve WGS and ART as experiments, and
...

Spencer

.

---

From: "Charles E. Perkins" <[email-masked]>
Date: Mon, 09 Feb 2004 00:14:50 -0800

> flawed in so many ways that I think it's unfixable.

I find this statement very offensive.

It basically demeans to the point of insult the
collective effort of subject-area experts.
Moreover, it ignores the reality that a working
group _needs_ such a document as an organizational
tool to sharpen and focus its efforts.  Otherwise
the progression of a long line of Internet Drafts
becomes a confusing mass to everyone, inside or
outside the group.

And, by the way, I don't remember putting forth
any part of the proposal about WGS as any kind
of "bypass" mechanism.  I guess it was a strawman
that was easy to knock down.

Regards,
Charlie P.
.

---

From: Margaret Wasserman <[email-masked]>
Date: Mon, 09 Feb 2004 06:42:40 -0500

Hi Spencer,

>(short of nomcom/AD recall).

I think that this type of process would alleviate my concerns, depending
upon how the ART is constituted.  If the ART contains people from all
different areas, then yes.  If it is all from one area, then I'd prefer
to see the WG identify a list of related areas and get reviews from
those areas, as well.

To take your example, the WG might decide that ipOverHamsterWheels
technology, as it involves a non-broadcast, asymmetrically routed
network with an unusual MTU size (or whatever), might have impact
on the congestion avoidance mechanisms in transport protocols.  There
may also be some unique security properties (is the hamster trusted
or not?).  So, the WG might seek reviews from IP experts, transport
area experts and security experts.  Since the WG may not be aware of
all of the impacts of their technology, the responsible AD might be
able to help the WG to identify areas where review is required.

In most cases, having the AD and WG chair both involved in the
sign-off should eliminate the possibility that one person is
deciding about his/her own document.

In general, ADs do not serve as the responsible AD for WGs that they
chair (some area WGs are exceptions), and I certainly think that an
AD could avoid a situation where s/he is all three of the editor,
chair and AD by asking another AD to make the decision.  This type of
recusal is quite common when ADs are involved in writing documents
in their own areas. For example, Thomas is shepherding a couple of
documents for the IPv6 WG (even though I am the responsible AD)
because I was involved in writing them.

I can also sympathize with the view that says that the AD shouldn't
have to be involved in approving a WGS, though (because the AD is
busy, not because I think we should be publishing WGS documents that
would never pass IESG review).

Perhaps an alternative would be to have an ART chair who could
approve the WGS (taking responsibility for ensuring that it
received proper cross-area review first), or the ART could vote
on whether to approve a WGS (2/3 required to do it?), or something
along those lines...

Margaret

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 9 Feb 2004 06:57:52 -0600

Hi, Pekka,

I've seen your assertion stated previously, that it is possible to
move to PS in a short period of time if everyone is moving. I think
your experience says quite a bit about your gifts in a number of
areas! Please take this as a compliment.

I'm thinking PS still has at least two problems that fast PS approval
doesn't fix, reference dependencies and IANA dependencies.

I don't know that I would suggest WGS if these were the only two
problems we encounter with PS, but I note that we haven't figured out
how to fix them yet, either.

I wonder (and no one will see this posting for a few days, because
we're all recovering from the initial draft cutoff weekend) if the
IETF controlled the STD numbering, and could assign STDs to RFCs and
APPROVED drafts, if this would fix most of the remaining issues.

For instance, external bodies could reference the draft by STD number,
instead of RFC number, so their needs would be met, etc.

But I'm thinking out loud, with not a lot of sleep. Others can correct
me.

Spencer

----- Original Message ----- 
From: "Pekka Savola" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, February 08, 2004 11:57 PM
Subject: Re: Using WGS to bypass external review (was: Re: [newtrk]

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 9 Feb 2004 07:09:43 -0600

Dear Margaret,

Thanks for your feedback.

http://www.ietf.org/internet-drafts/draft-zinin-early-review-00.txt
seems to be headed for ARTs that are composed of representatives from
multiple areas.
http://www.ietf.org/internet-drafts/draft-iesg-alvestrand-twolevel-00.txt
proposes one ARTist per area.

So, (discussing published proposals is SO much more productive!), we
seemed to be headed for ARTs that are multi-area, not single-area.

http://www.ietf.org/internet-drafts/draft-iesg-alvestrand-twolevel-00.txt
also proposed an "area supervisor", who I don't think was an AD as we
know ADs today, who could get involved without sucking up AD time...

Spencer

p.s. I just noticed that
http://www.ietf.org/internet-drafts/draft-iesg-alvestrand-twolevel-00.txt
was published more than a year ago? I feel sick! We need to be moving
so much faster than this...

----- Original Message ----- 
From: "Margaret Wasserman" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, February 09, 2004 5:42 AM
Subject: Re: Using WGS to bypass external review (was: Re: [newtrk]

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 9 Feb 2004 07:13:03 -0600

Sigh...

From: "Spencer Dawkins" <[email-masked]>
To: <[email-masked]>
Sent: Monday, February 09, 2004 7:09 AM
Subject: Re: Using WGS to bypass external review (was: Re: [newtrk]
last call on newtrk draft charter)

> proposes one ARTist per area.

... "on each Area Review Team" ...

Spencer

.

---

From: Pekka Savola <[email-masked]>
Date: Mon, 9 Feb 2004 15:28:56 +0200 (EET)

> doesn't fix, reference dependencies and IANA dependencies.

That's true, of course.

IANA dependencies is a thorny problem.  That is, many folks implement 
specs way before they have been approved, much less have been sitting 
in the RFC-editor's queue, waiting for IANA allocations (or waiting 
for IANA).  Until the IANA allocations have been made, implementations 
just "steal" the numbers or use some experimental numbers, making it 
dificult to create interoperable implementations.  And when the spec 
is published, the code needs to be changed.

IMHO, this seems to call for early IANA allocations, made at the stage 
where the technology has matured enough so that folks know what they 
want.  This can be achieved by RFC 2434 'IESG Approval' process (for 
example), but has obvious problems particularly in 'Standards Action' 
process.  This might warrant some flexibility.  One proposal has been 
made, at least: draft-kompella-zinin-early-allocation-00.txt.

As for the reference dependencies, I don't see this as a huge issue.  
That's because the specification is already stable/unchanged, just
waiting for the references to come out.  It can already be implemented
etc. -- the only thing that's missing is RFC number (which may or may
not be critical :) -- and the potential knowledge of IANA allocations,
unless you can check them yourself from the IANA web page.  However,
if the normative references are yet so unstable that you can't
implement/understand the spec without them, you're of course still in
for a trouble.

In many cases, the worst cases come when people haven't understood 
what Normative References mean wrt. publication.  In some other cases, 
the documents which are heavily interdependent should be published 
together in most cases in any case -- if major modifications are made 
to the other at the last stages of the process, those might warrant 
modifications to the other one as well -- and that's already too late 
if it has been published (or approved, really).   So, this seems to be 
more of an education issue to me.

.

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Feb 2004 08:40:26 -0500

> collective effort of subject-area experts.

No it doesn't, or at least, that's not the intent.  It just recognizes that
most working groups affect people outside their subject area, and therefore,
that subject-area experts _by themselves_ are not qualified to decide what
should be a stanard.

> tool to sharpen and focus its efforts. 

No, it's recognizing the reality that sharpening and focusing the efforts
of a working group prior to external review is investing too much effort
in a design that is likely to be flawed, and which will be difficult to 
fix once the flaws are discovered.

There's nothing at all wrong with a WG saying "we have tentative consensus
on this set of design goals" or "we have tentative consensus on this
rough model for a design" and getting external reviews on those.  Nor is 
there anything wrong with a WG saying "having received feedback on our
set of design goals and rough model, here's a first cut at a detailed
protocol spec that we think is workable" and soliciting feedback on that.

But if the first WG document that gets external review is one where the 
WG says "we think we're done with this detailed specification and it 
should be ready to implement" -- well, basically that's the situation we
have now, except that IESG gets to review things.  Well guess what?
By this point WGs are't willing to take external feedback - they don't 
want to hear that they haven't done a good job with security or that 
their solution doesn't scale or that they are harming the ability of
other protocols to work well.  By this time the WG has labored too long
without external clue, it has built up confidence in its work, vendors 
are planning to ship product within  weeks (if they haven't started already)
that are based on this spec, etc.  It's too late for external feedback,
whether from IESG or anybody else.  And when the WG does get negative feedback
from IESG it takes months to fix, maily because the WG participants simply
aren't willing to admit that significant changes are needed to address valid
concerns that they didn't think of first.

For the external feedback to be valuable, it has to happen  earlier.
And _that_, in a nutshell, is what is wrong with -twostage-.

.

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Feb 2004 11:08:39 -0500

> > It's completely broken for a WG to declare consensus on anything
> > resembling a complete spec that hasn't enjoyed significant external
> > review.  What this means is that the WG has become too committed to
> > details before it has had a chance to get clue input from outside the WG. 
> > That's a big part of what is wrong with our current process, and
> > draft-dawkins-pstmt-twostage would only make it worse.
> 
> that's good enough for a WGS"? 

No.  You need to separate the concept of "working group snapshot" -
which is a useful concept - from the concept of "ready for implementation".
The two do not go together.

> FWIW, I am not opposed to reviews, only slow reviews.

Good reviews take time.  I think that reviews will take less time if
there is supporting evidence that the WG has done its homework.  But in
general, if you want faster reviews, find a way to write better
specifications.

> permitted, not discouraged.

Such health warnings are worse than useless.  Most of the time when a
spec is bad you need a lot more than a hundred words to explain why 
it is bad and how to use it safely if it is possible to do so.  That's one of
the reasons that IESG reviews can take so long - IESGers know that WGs will
balk at suggestions that large changes are necessary, so they are trying to 
figure out how to solve the problem with small changes.  The solution is not
to try to reduce IESG's role to making even less input, it's to force
WGs to identify these problems much earlier in the process.  

> reviewers are all in the weeds. 

I don't know where you got this idea, but it doesn't resemble anything
that I said.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Feb 2004 11:26:09 -0500

> - WG chair requests reviews from Area Review Team

if you get to draft -09 before asking for area review, chances are good
that you're already too far off into the weeds.  what you need is to get
external review MUCH earlier, long before fundamental design decisions
are carved in stone. the errors that WGs are making are not in the details,
they're at more fundamental levels.

(and when errors in detail are made, they're often easy to fix - it's 
fundamental design errors that aren't easy to fix)

also, for many groups, "Area Review" isn't nearly broad enough.  what I 
think we'd need to do is to have IESG specify, on a per-group or 
per-protocol basis, how widely reviewed a proposal needs to be.

.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 09 Feb 2004 15:24:58 -0800

> seemed to be headed for ARTs that are multi-area, not single-area.

I don't think they're that different - draft-zinin proposed that a draft be 
reviewed by ARTs from multiple teams, each team being associated with an 
area, while draft-iesg-alvestrand proposed that a draft be reviewed by ARTs 
from multiple areas, but that the same ARTs always worked together in a 
team. Both had drafts reviewed with multiple "area glasses" on.

> know ADs today, who could get involved without sucking up AD time...

actually the area supervisor was originally a new name for the AD in "my" 
proposal, but at some point it was pointed out that it may be useful for 
them to be different....

> so much faster than this...

Apologize very much for making the process look worse than it is....
draft-iesg-alvestrand-twolevel was last edited on October 17, 2003 and 
announced on October 28, 2003. That's still only 4 months ago.
I was in such a hurry to get it out that I didn't notice that the date was 
left over from the draft I copied the XML from.....!

                      Harald
.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 9 Feb 2004 18:53:55 -0600

Hi, Keith,

I'm not sure I understand what you mean by "implementation" - do you
mean writing code, or deploying code in networks?

Spencer

----- Original Message ----- 

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 9 Feb 2004 18:59:03 -0600

Ah - Keith, you couldn't tell it from what I typed, but I didn't mean
to imply that the woozle working group produced nine versions of the
draft before asking for ART input.

I agree with the need for both early ART review (to address the
problem you're talking about) and pre-publication ART review, but
didn't clearly say that I was talking about pre-publication ART
review, in this example.

Thanks for picking this up.

Spencer

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Spencer Dawkins" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, February 09, 2004 10:26 AM
Subject: Re: Using WGS to bypass external review (was: Re: [newtrk]

.

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Feb 2004 20:00:49 -0500

I mostly mean writing code that goes into products.  But in many cases we need
"working group snapshots" and external review long before we produce a spec 
that is detailed enough to implement.  

> > which is a useful concept - from the concept of "ready for
> > implementation".  The two do not go together.
.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Feb 2004 20:29:48 -0800

>> collective effort of subject-area experts.

KM> No it doesn't, or at least, that's not the intent.  It just recognizes that
KM> most working groups affect people outside their subject area,

This exchanges highlights a misunderstanding about the concept of a
Working Group Snapshot.

First of all, let's remember that working groups are open.  Anyone with
serious interest in a topic is welcome to participate.

That does not mean there is not need for external review and support,
outside the working group, but it is a reminder that working groups are
not supposed to be elitist cabals.

Second of all, we would all do well to remember that working groups
declare consensus frequently, over the course of their work.  It is the
nature of the working group process to do this.  It is the way that
incremental progress is made on specifications.

The concept of WGS is simply to formalize snapshots of these sets of
working group consensus decisions, in order to facilitate coordination
of related efforts -- such as testing -- prior to seeking or achieving
IETF-wide review and approval.

In other words, WGS is really a small, pragmatic idea, which is
unfortunately being cast as something bloated or deceptive or "final".

KM> There's nothing at all wrong with a WG saying "we have tentative consensus
KM> on this set of design goals" or "we have tentative consensus

When a working group achieves rough consensus on something, it is not
"tentative consensus".  It is rough consensus, as in "we have resolved
this point, now let's move on."  Sometimes a working group chooses to
re-visit a topic, but that is supposed to be the exception.  (The fact
that it happens regularly is a matter of separate concern.  Forward
process dictates not doing it often.)

Yes, comments from within or without the working group might prompt the
re-visiting.  That's fine.

But the decision is not "tentative".  The IETF does not require everyone
to agree.  A working group does not have to satisfy everyone with
concerns.

It needs to achieve rough consensus.  And it needs for no one to come up
with a show-stopper that indicates the decision won't work.

The idea that anyone with any objection is empowered to force a working
group to reverse itself is what is broken.  And, yes, so is the idea
that the working group can ignore valid, serious criticism.

But all of this is quite different from a condition that can be
characterized as "tentative decisions".

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 9 Feb 2004 20:35:31 -0800

Spencer,

SD> - the AD thinks -10.txt makes sense (given the reviews and the
SD> explanation of why the third thing is not a problem) and says "include
SD> the words 'This Internet Draft is issued as a Woozle Working Group
SD> Snapshot' in -11.txt.

I think this is exactly the wrong line of thinking for WGS.  It makes
the label that much more meaningful, and we need to make sure the label
is useful, but not all that meaningful.

If, in fact, WGS is for the purposes of working group "coordination",
such as testing, then it needs to be subject only to working group
consensus, just like any other working group decision.

The instant we start invoking requirements for external review or area
director approval, or any other Big Process step, we have bloated the
label into more than is useful.

d/

.

---

From: Keith Moore <[email-masked]>
Date: Tue, 10 Feb 2004 09:36:35 -0500

> KM> No it doesn't, or at least, that's not the intent.  It just recognizes
> KM> that most working groups affect people outside their subject area,
> 

Whose concept of a WGS?  Yours, mine, or the concept held by the author of a
particular I-D?
 
> IETF-wide review and approval.

That - taken by itself - is a fine thing, and I have no problem with that.
It's the association of this idea with the idea that the WG is going to
develop the protocol in significant detail before getting external review
that I object to.

(and if that isn't how _you_ anticipate seeing WGS used, then you and I
are not in disagreement...at least about this :)
 
> unfortunately being cast as something bloated or deceptive or "final".

I agree that this casting is unfortunate.  
 
> KM> There's nothing at all wrong with a WG saying "we have tentative
> KM> consensus on this set of design goals" or "we have tentative consensus
> 
> "tentative consensus". 

That's one of the things that needs to change.  If the consensus has not had
external review, it needs to be tentative consensus, because the possibility
exists that it will need to be revisited in light of feedback from external
review.

> to agree.  A working group does not have to satisfy everyone with

The working group IS required to meet the requirements of 2026, among which
are "no known technical omissions".  And the mere "concern" of someone
external to the working group should have as much (or little) weight as the
"concern" of someone within the working group.  It should not require a
"showstopper" from an external party.

For any standards decision, we need to reach consensus of the entire
community, not just of the working group.  We have had too many working groups
effectively isolating themselves (by various means) to try to get a standard
published  which was harmful to some other constituency.

> that the working group can ignore valid, serious criticism.

Working groups have a tendency to dismiss valid, serious criticism from
external parties as "anyone with any objection".

> characterized as "tentative decisions".

Actually "tentative decisions" seems exactly right.  EVERY WG decision is
tentative pending external review.  What we need to do is to get the
external review for fundamental decisions to happen earlier in the process,
so that objections can be addressed before it's too expensive and time
consuming to do so.

Keith
.