Subject: Re: Response requested: Proposal regarding milestones

From: Ted Hardie <[email-masked]>
Date: Thu, 19 Dec 2002 08:09:18 -0800 (PST)

Bert,
	Sorry for my poor writing; I had meant to indicate "no
objection" was a minimum response, not the common response.  I agree
with you that it would be much better to have working group
members make definite statements about their views on the
document.
			regards,
				Ted Hardie

> 
> Ted,
> 
> What I like about your strawman is that it (at least tries to)
> creates the idea with WG participants that they MUST review
> documents and that they MUST comment on them. That is, they
> cannot just be silent, they must also post a positive statement
> if they have no objection.
> 
> I would change the "no objection" into a few categories 
>   - I read current document and have implemented code and agree
>     that it is a good document and a usefull protocol
>   - I read current document and agree that it is a good document
>     and a usefull protocol
>   - I read current document and have no objection
>   - I read an earlier version and have no objection
>   - I did not read it, but habve no objection
> Well,, you can think of more. I guess you get the gist.
> They are details, but the good thing I see in that is that we get
> a much better feel of how well documents have been reviewed. And
> by having to explicitly state that they read the current version
> and that they agree, exposes the participants support better
> to the community to, so they would be inclined to feel more
> responsible about their statement.
> 
> Thanks,
> Bert 
> 
> > -----Original Message-----
> > From: Ted Hardie [mailto:[email-masked]]
> > Sent: donderdag 19 december 2002 9:30
> > To: [email-masked]
> > Cc: [email-masked]; [email-masked]
> > Subject: Re: Response requested: Proposal regarding milestones
> > 
> > 
> > > 
> > > At 02:31 PM 12/12/2002 -0800, Ted Hardie wrote:
> > > >this trio of influences (pulses in attention, tension 
> > between timeliness 
> > > >and quality, and tension between the voluntary nature of 
> > participation and 
> > > >the "contract" method of management) is very common.
> > > 
> > > True enough, and timeliness is only one consideration. But 
> > it is one that 
> > > we routinely do poorly on, and the lack of it is a symptom 
> > of far deeper 
> > > problems. I merely propose that we treat it like we treat a 
> > fever: find the 
> > > cause of the symptom and decide what we're going to do 
> > about it, but not 
> > > ignore it.
> > > 
> > > >I agree that this conversation needs to happen, but I 
> > don't think this is 
> > > >a good lever.
> > > 
> > > What lever would you suggest? The right lever has two 
> > characteristics, I 
> > > will argue: it is sufficiently unavoidable and problematic 
> > to actually act 
> > > as a lever, and sufficiently benign that, having served its 
> > purpose, the 
> > > working group can continue without a noticeable hiccup.
> > > 
> > > My concern with the 18 month rule and the N working group 
> > rule are, besides 
> > > being arbitrary, they fail the second consideration. 
> > 
> > Believe it or not, I've been thinking about Fred's question a lot over
> > the week since the original exchange, and I think it points to a 
> > fundamental problem (which may be one of the causes of this symptom-I
> > think there may be several).  
> > 
> > Basically, it is very difficult to come up with appropriate tools for
> > managing working groups because it is very difficult to define working
> > groups.  Not in the charter-and-milestones sense, but in the sense of
> > "who has taken on the work".  As I tried to come up with answers to
> > Fred's questions about the right tool, I realized all of the ones I
> > had applied mainly to the wg chairs (and occasionally document
> > authors).  This was, quite simply, because they were the only folk who
> > could truly be identified with the working group in an unambiguous
> > way.
> > 
> > We currently define working groups by the mailing list noted in the
> > charter.  We can identify participants on the mailing list; those who
> > express opinions, submit documents, or provide critiques.  The process
> > as defined is remarkably open and it has the tremendous benefit that
> > anyone can make a comment and be heard.  That openness, though, makes
> > it difficult to make anyone other than the wg chairs and current
> > authors accountable for the working group making progress.  Making a
> > comment on a document does not, in essence, imply that you are taking
> > responsibility for the work of the working group.  That ambiguity, in
> > turn, makes it very difficult to apply any management-by-objective
> > tools to the working group itself.
> > 
> > I've been thinking about what we might do that would retain the
> > openness of the current process while creating some sense of who has
> > taken on the work of the working group.  Below is a strawman proposal.
> > 
> > (Note that I am not sending this message to problem-statement right
> > now because of this strawman--they are meant to think about the
> > problem, not solutions.  If others agree that the problem I've defined
> > above is one of the issues, I will send that)
> > 
> > Proposal:
> > 
> > 1)  All working groups are open to participation by anyone and
> > anyone may join a working group at any time.
> > 
> > 2) Joining a working group adds you to the mailing list for
> > the working group.
> > 
> > 3) Anyone may join a working group mailing list without becoming
> > a member of the working group.
> > 
> > 4) When joining a working group, you agree to read the documents
> > produced for the working group:  mailing list, minutes of meetings,
> > chartered drafts.
> > 
> > 5) For each charter document, you agree to provide public feedback
> > within the time frames set out by the chair (i.e. if a two week
> > working group last call is issued, within two weeks).  This can be "no
> > objection" if the documents are okay by you.  This feedback would be
> > in the form of email to the mailing list.
> > 
> > 6) If you consistently do not provide feedback within the timeframes
> > set out, you are (may be?) dropped as a working group member.  This
> > does not drop you from the mailing list.
> > 
> > 7) Rejoining a working group after being dropped requires WG chair
> > approval.  Note again that dropping someone from membership
> > does not restrict their ability to provide comment (see above).
> > 
> > 8) When chartering new work, ADs may ask for a list (to avoid
> > cronyism, a number?) of those who have indicated willingness to become
> > working group members.
> > 
> > 9) In reviewing working groups, ADs may ask for a current list of
> > working group members in order to assess the energy and expertise
> > available for the existing or new work.
> > 
> > 10) Anyone may raise a technical issue with the work produced by a
> > working group.  Working group members have no special rights in this
> > regard. They have, instead, taken on a responsibility to consider and
> > answer technical problems raised.
> > 
> > 11) In assessing consensus, working group chairs would consider first
> > if there are technical objections raised by anyone, then if they had
> > received sufficient positive input from the membership to proceed.
> > 
> > So, becoming a working group member requires committing to a lot of
> > work without getting much in the way of benefits.  There would
> > obviously need to be some form of potential recognition (in documents,
> > perhaps, or on a working group web page).  It might also become easier
> > to justify time spent to an employer if the commitments were more
> > explicit.  The primary reason for taking on membership, though, would
> > have to be a desire to see the work complete.  To make that desire
> > translate into a willingness to publicly record one's engineering
> > opinion about specific proposals, it would have to be clear that a
> > lack of committed members meant that the work would not complete.
> > 
> > I suspect that this would expose something that is already true: many
> > working groups have small numbers of active participants and large
> > numbers of auditors.  By making that explicit without detracting from
> > anyone's ability to raise technical issues, we might improve the
> > ability of working group chairs, ADs, and other working groups to
> > identify who has taken on specific work.  That, in turn, might help us
> > with making progress using management by objective tools.  There is no
> > guarantee, of course, because these are still volunteers, but it would
> > at least help us know one of the most basic elements of the equation.
> > 
> > Of course, the scary flip side to that possibility is that this step
> > creates a regime in which de facto voting by those with narrow
> > interests could ride roughshod over good design.  That prospect
> > almost caused me to scrap even discussing this proposal.  I believe,
> > however, that the fundamental problem is real even if this proposal
> > is dangerous, and I hope having both the problem and the proposal
> > will help frame how this particular problem fits into the larger
> > set being discussed elsewhere.
> > 				regards,
> > 					Ted Hardie
> > 
> > 
> > 
>

---

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Thu, 19 Dec 2002 12:24:32 +0100

Ted,

Thanks,
Bert

---

From: "James Luciani" <[email-masked]>
Date: Thu, 19 Dec 2002 13:50:39 -0500

I do not necessarily have a problem with this, but it sounds like what you
describe below is voting by another name.

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 19 Dec 2002 14:02:28 -0500

>describe below is voting by another name.

I know that this may be a heretical question, but...

What is so bad about voting?

Margaret

---

From: Marshall Rose <[email-masked]>
Date: Thu, 19 Dec 2002 11:18:43 -0800

> What is so bad about voting?

"we reject kings, presidents, and voting. we favor only rough concensus and
running code."

voting introduces a host of problems, not the least of which is having to
qualify the voters and having to deal with election fraud.

here is one question that illustrates the problem: should voting be by
individual or organization? if by individual, then how do we prevent large
companies from stuffing the ballot box? if by organization, then why are we
penalizing large companies that happen to have attracted senior talent?

voting works great for electoral systems which must balance a host of competing
social issues. it doesn't work so great when the rules of physics are involved.
just because a certain state legislature in the US passed a bill that redefined
a certain transcendental number to be exactly "3", doesn't make it "3"...

/mtr

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 19 Dec 2002 13:40:45 -0800 (PST)

> qualify the voters and having to deal with election fraud.

I certainly agree with this. However, voting also has one salutory
feature: it actually provides an incentive for participants to read the
specifications!

What fraction of IETF participants actually bother to do that nowadays?
At IETF 55, it seemed to me that fraction was smaller than I'd ever seen
it.

A "rough consensus" of people who haven't read the documents is not very
interesting. It also places an unfair burden on the IESG, who actually do
read the documents.

So before we get too smug about the importance of our Values and
Principles, it might be worth reflecting on how we can retain those things
while also adding another feature: Accountability.

One (frightening) job description of the IESG is "the people who
actually commit to reading IETF specifications and who are accountable
if they don't make sense."

In organizations that endeavor to improve quality, this job description is
supposed to apply to everyone. And one way that particular principle is
furthered in other SDOs is by dangling the carrot of voting rights.

---

From: Marshall Rose <[email-masked]>
Date: Thu, 19 Dec 2002 14:56:33 -0800

> feature: it actually provides an incentive for participants to read the

err, there's no causation here. in fact, idon't there's even correlation.

> while also adding another feature: Accountability.
    
as the co-author of the PACT document, it's hard to argue with that...

> if they don't make sense."

this isn't a bad definition; and, (setting aside my "no surprises"
exhortation), i'm perfectly happy with the process whenever the iesg
comes back (in a timely fashion) and clearly states why something is broken.
i'm even happier when they explain what it will take to fix it (though i
realize that sometimes things are so broken as to make this problematic.)
    
i suspect that many wg participants share this view.

> furthered in other SDOs is by dangling the carrot of voting rights.

err, uh, i'll say it again: there is no correction between voting rights
and having read/understood/analyzed the thing you're voting on.
    
ten years ago, i used to joke that we ought to require a passphrase
before letting people show up at a wg meeting. the passphrase would be a
non-sensical sentence buried in the middle of one of the I-Ds on the
agenda.  no passphrase, no entrance.
    
so, i can give you the value-add of voting, without any voting... simply
implement my nonsenical idea from 1992...
    
/mtr

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 19 Dec 2002 17:56:16 -0500

>interesting. It also places an unfair burden on the IESG, who actually do

Even worse, it becomes a popularity contest where an effective
presenter can create a "consensus" for mediocre or even harmful
ideas.

>if they don't make sense."

Pretty scary, but true...

What are our options for pushing accountability down into lower
levels of the organization?

Margaret

---

From: "David Barr" <[email-masked]>
Date: Thu, 19 Dec 2002 17:52:55 -0600

> "we reject kings, presidents, and voting. we favor only rough 
> concensus and running code."
> 
> voting introduces a host of problems, not the least of which 
> is having to qualify the voters and having to deal with 
> election fraud.

Regularly in the USEFOR (Usenet message format) group we have
successfully overcome big obstacles and achieved consensus through
(admittedly often complex) voting.  The big ones were done with "instant
runoff" style voting, with the added twist of specifying your tolerance
level.  For example you could vote in favor of D, A, or B (in that
order), but say that C was unacceptable to you.

USEFOR by its nature is probably largely unique, and voting there has
helped us a great deal, though we do use it as a last resort.  Our
qualities include:

- The subject matter is Usenet, which is an existing and complex
"protocol" that has to interoperate with a number of different
standards, namely email.  Usenet, as we've found, often has issues
unlike any other standard out there, even email.
- The proposals to move Usenet forward are many (the biggest being
having a plethora of ways and degrees with which to implement 8-bit
characters).  Sometimes it was simply more important just to pick
something rather than drag on forever trying to find a "perfect"
solution (even though we all realized one didn't exist).
- To a certain degree there are in fact too many proposals and
possibilities on the table. 
- To my knowledge there haven't ever been any vendor axes grinding.  The
only "vendors" participating are those who are a couple developers of
free software packages.  The axes they may grind are viewed as entirely
their own, not of any company.
- People trust voting to our editor (who does the tallying), and voter
fraud hasn't ever been an issue.  There really isn't any motivation for
fraud.
- Because there are no vendors involved, however, we never meet in
person at IETF meetings because there's no funding.  This is a big
obstacle to consensus.

While I truly think of voting as a last resort, in our case I can
honestly say the group would most likely have collapsed upon itself if
we didn't take a vote now and then.  The discussions often get so heated
and drawn out we lose all sense of what the actual feelings are of the
entire group.

I can definitely see the pitfalls of voting, but in our case we really
didn't have to worry about them.

--Dave
USEFOR chair

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 19 Dec 2002 19:10:41 -0500

>I'm not certain I'm crazy about voting in the "there were 21 working group 
>members and 50%+1 consisting of Tom, Dick, ..., and Harriet agreed so 
>we're doing it my way" sense - "90% in favor and no strong objections" 
>sounds more like it.

Depending on your definition of "voting", I may not be in favor
of it at all...

I am certainly not in favor of a system where a 51% majority
makes every decision.

However, I do think that doing a rough "vote", and finding out
(roughly) how many people support each of several positions can
help the chairs to decide how to guide the WG to consensus.

I also think that there are cases where the WG should reach
consensus on the _process_ that will be used to make a decision
between several options, none of which would ever (in an of
itself) achieve consensus.  This may lead to something similar
to what Melinda described in midcom.

Margaret

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 19 Dec 2002 19:20:24 -0500

>
>I certainly saw a careful count of raised hands in Atlanta over
>the V6 site-local issue, so on the one (raised) hand we already have
>some form of expression of group consensus by simple numbers
>(voting) happening already in some working groups.

I wouldn't describe what we did as a "careful" count (I know,
because I was doing some of the counting), but we did count.

When there are about 150 people in the room expressing an opinion,
I am not, personally, capable of remembering an "impression" of how
many people raised their hands for one choice, and fairly comparing
that with my "impression" of how many people raised their hands for
another choice.

Hums don't work in a big room, either.  I can't hear someone yelling
from the back of the IPv6 WG without a microphone, so I am certain
that I won't be able to hear them humming...

>The original positioning of the voting issue was one of the
>IETF attempting to take some form of technical high ground
>in comparison to ITU-T formal ballots, where the task is not
>to create technically superior outcomes, but to create outcomes
>that will engender a majority of the votes. As the IETF was always
>keen to point out at the time, such outcomes were often technically
>inferior to about anything else you may care to think of.

And, you don't think we make the same type of compromises to
get to " rough consensus"?

Sometimes in these discussions I wonder (seriously) about whether
we all work in the same organization.  I think that different
techniques and practices may occur in different areas of the
organization.  That may be okay, but it does make it difficult
to have a coherent discussion of how to change our existing
practices.

>But if you are asking for a sense of what is a superior technical
>approach then voting is often a very misleading tool.

I agree.

So, which of the following jobs would you assign to a WG chair:

         (1) Drive the process to produce the best technical
                 result, and send that result to the IESG, or
         (2) Manage a process that is driven by the consensus
                 of the WG, make sure that the documents
                 pass ID-nits, and send the consensus result
                 to the IESG?

I used to think that we were supposed to do (2), but I've recently
been told by my ADs that I should be doing more of (1).  Maybe
the correct balance is somewhere in between?

Margaret

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 19 Dec 2002 16:02:57 -0800 (PST)

> to have a coherent discussion of how to change our existing

I think that views of what the IETF is (and should be) vary fairly
dramatically based on how long people have been involved.

Just for grins, I went back into the IETF archives, and was startled to
find some WG milestones that involved writing a standards track document
and getting to RFC status within 6-9 months. The minutes described a
tightly written charter, a motivated group that involved both vendors and
customers and the milestones came close to being met in practice. It was a
bit like reading about the building of the Grand Coulee Dam, and wondering
how it could have possibly been completed as quickly as it was.

Now, I'm sure that those archives also include their own share of
disasters -- but I was struck by the difference in expectations. Clearly
in past years, there was an expectation that, at its best, the process
could work quickly -- and results did justify those expectations to some extent.

If you happen to remember those days (even faintly), it
colors your expectations about how the IETF can perform.

>                  pass ID-nits, and send the consensus result

Having experienced WG chairs who saw themselves in both roles, the
decision does not appear that clear cut to me.

Today's "nutty new idea" is tommorrow's orthodoxy. And looking back, there
are more than a few examples of ideas that were deemed "inappropriate"
that ended up becoming pervasive.

It's worthwhile remembering, for example, that many of the most important
protocols did not originate in the IETF, but were brought to it for
standardization.

So there is some amount of logic behind the desire for less dictatorial WG
chairs -- and everything considered, I don't know that I would necessarily
trade today's approach for yesterday's.

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 19 Dec 2002 21:30:40 -0500

At 05:45 PM 12/19/2002 -0800, Fred Baker wrote:
>At 07:10 PM 12/19/2002 -0500, Margaret Wasserman wrote:
>>I do think that doing a rough "vote", and finding out
>Oh, well, we *do* that. It's just that gauging a consensus is not about 
>whether 89% or 90% agree with it.

I completely agree.

Margaret

---

From: Fred Baker <[email-masked]>
Date: Thu, 19 Dec 2002 14:12:39 -0800

>What is so bad about voting?

One has to decide who may vote. The IETF has been open-ended in a number of
regards, making this problematic.

In Ted's proposal, there would be a clear understanding of that, and one
could benefit from an ability to use what other bodies describe as a "no"
vote: "no, but if you make <list of edits> I will change my vote to 'yes'."

---

From: Fred Baker <[email-masked]>
Date: Thu, 19 Dec 2002 15:20:47 -0800

>levels of the organization?

mine is obviously to make the working groups, and therefore the people in
them, accountable for what they produce. I have several reasons: it seems
"right", it offloads the IESG, it seems to reinforce the ethic we claim
applies to us in the first place, and so on.

I'm not certain I'm crazy about voting in the "there were 21 working group
members and 50%+1 consisting of Tom, Dick, ..., and Harriet agreed so we're
doing it my way" sense - "90% in favor and no strong objections" sounds
more like it.

---

From: Geoff Huston <[email-masked]>
Date: Sat, 21 Dec 2002 05:16:53 +1100

On the other hand there remains the issue that there are
all kinds of voters and all kinds of questions, and if the task is to
create standards relating to functional and useful technologies then
often the search is for a formation of consensus that supports
a technically superior approach. This is not the same as a vote.

So if you are asking a group for an informed view about the
level of consensus behind one of a number of viable and roughly
equivalent approaches, then humming, show of hands, etc are all
tools that help the group to understand where there is common
ground. In this sense voting, if that's what you want to call a rough
count of a sea of hands, is as good a tool for any other.

regards,

     Geoff

---

From: Geoff Huston <[email-masked]>
Date: Sat, 21 Dec 2002 06:56:49 +1100

>the correct balance is somewhere in between?

There is a balance, yes, and our ethos and values attempt to
say [1], while a pragmatic approach tends to suggest that
elements of [2] are entirely natural and reasonable.

Geoff

---

From: Fred Baker <[email-masked]>
Date: Thu, 19 Dec 2002 17:45:33 -0800

---

From: "James Luciani" <[email-masked]>
Date: Fri, 20 Dec 2002 13:29:17 -0500

I think she asked for "options" not "reasons".

> mine is obviously to make the working groups, and therefore the people in

Are you sure it will ultimately offload the IESG?  Won't the IESG have to
still do as much diligence?
Also, of course, feeling right is subjective (*genuinely no offence
intended*)
Could you elaborate on the reinforce ethics part?

> applies to us in the first place, and so on.

I would observe that other groups have made an effort at meritocracy and
also had voting as a method of "divining consensus"