Subject: Re: Response requested: Proposal regarding milestones

From: Ted Hardie <[email-masked]>
Date: Thu, 19 Dec 2002 00:30:16 -0800 (PST)

> 
> At 02:31 PM 12/12/2002 -0800, Ted Hardie wrote:
> >this trio of influences (pulses in attention, tension between timeliness 
> >and quality, and tension between the voluntary nature of participation and 
> >the "contract" method of management) is very common.
> 
> True enough, and timeliness is only one consideration. But it is one that 
> we routinely do poorly on, and the lack of it is a symptom of far deeper 
> problems. I merely propose that we treat it like we treat a fever: find the 
> cause of the symptom and decide what we're going to do about it, but not 
> ignore it.
> 
> >I agree that this conversation needs to happen, but I don't think this is 
> >a good lever.
> 
> What lever would you suggest? The right lever has two characteristics, I 
> will argue: it is sufficiently unavoidable and problematic to actually act 
> as a lever, and sufficiently benign that, having served its purpose, the 
> working group can continue without a noticeable hiccup.
> 
> My concern with the 18 month rule and the N working group rule are, besides 
> being arbitrary, they fail the second consideration. 

Believe it or not, I've been thinking about Fred's question a lot over
the week since the original exchange, and I think it points to a 
fundamental problem (which may be one of the causes of this symptom-I
think there may be several).  

Basically, it is very difficult to come up with appropriate tools for
managing working groups because it is very difficult to define working
groups.  Not in the charter-and-milestones sense, but in the sense of
"who has taken on the work".  As I tried to come up with answers to
Fred's questions about the right tool, I realized all of the ones I
had applied mainly to the wg chairs (and occasionally document
authors).  This was, quite simply, because they were the only folk who
could truly be identified with the working group in an unambiguous
way.

We currently define working groups by the mailing list noted in the
charter.  We can identify participants on the mailing list; those who
express opinions, submit documents, or provide critiques.  The process
as defined is remarkably open and it has the tremendous benefit that
anyone can make a comment and be heard.  That openness, though, makes
it difficult to make anyone other than the wg chairs and current
authors accountable for the working group making progress.  Making a
comment on a document does not, in essence, imply that you are taking
responsibility for the work of the working group.  That ambiguity, in
turn, makes it very difficult to apply any management-by-objective
tools to the working group itself.

I've been thinking about what we might do that would retain the
openness of the current process while creating some sense of who has
taken on the work of the working group.  Below is a strawman proposal.

(Note that I am not sending this message to problem-statement right
now because of this strawman--they are meant to think about the
problem, not solutions.  If others agree that the problem I've defined
above is one of the issues, I will send that)

Proposal:

1)  All working groups are open to participation by anyone and
anyone may join a working group at any time.

2) Joining a working group adds you to the mailing list for
the working group.

3) Anyone may join a working group mailing list without becoming
a member of the working group.

4) When joining a working group, you agree to read the documents
produced for the working group:  mailing list, minutes of meetings,
chartered drafts.

5) For each charter document, you agree to provide public feedback
within the time frames set out by the chair (i.e. if a two week
working group last call is issued, within two weeks).  This can be "no
objection" if the documents are okay by you.  This feedback would be
in the form of email to the mailing list.

6) If you consistently do not provide feedback within the timeframes
set out, you are (may be?) dropped as a working group member.  This
does not drop you from the mailing list.

7) Rejoining a working group after being dropped requires WG chair
approval.  Note again that dropping someone from membership
does not restrict their ability to provide comment (see above).

8) When chartering new work, ADs may ask for a list (to avoid
cronyism, a number?) of those who have indicated willingness to become
working group members.

9) In reviewing working groups, ADs may ask for a current list of
working group members in order to assess the energy and expertise
available for the existing or new work.

10) Anyone may raise a technical issue with the work produced by a
working group.  Working group members have no special rights in this
regard. They have, instead, taken on a responsibility to consider and
answer technical problems raised.

11) In assessing consensus, working group chairs would consider first
if there are technical objections raised by anyone, then if they had
received sufficient positive input from the membership to proceed.

So, becoming a working group member requires committing to a lot of
work without getting much in the way of benefits.  There would
obviously need to be some form of potential recognition (in documents,
perhaps, or on a working group web page).  It might also become easier
to justify time spent to an employer if the commitments were more
explicit.  The primary reason for taking on membership, though, would
have to be a desire to see the work complete.  To make that desire
translate into a willingness to publicly record one's engineering
opinion about specific proposals, it would have to be clear that a
lack of committed members meant that the work would not complete.

I suspect that this would expose something that is already true: many
working groups have small numbers of active participants and large
numbers of auditors.  By making that explicit without detracting from
anyone's ability to raise technical issues, we might improve the
ability of working group chairs, ADs, and other working groups to
identify who has taken on specific work.  That, in turn, might help us
with making progress using management by objective tools.  There is no
guarantee, of course, because these are still volunteers, but it would
at least help us know one of the most basic elements of the equation.

Of course, the scary flip side to that possibility is that this step
creates a regime in which de facto voting by those with narrow
interests could ride roughshod over good design.  That prospect
almost caused me to scrap even discussing this proposal.  I believe,
however, that the fundamental problem is real even if this proposal
is dangerous, and I hope having both the problem and the proposal
will help frame how this particular problem fits into the larger
set being discussed elsewhere.
				regards,
					Ted Hardie

---

From: Fred Baker <[email-masked]>
Date: Thu, 19 Dec 2002 06:59:31 -0800

>above is one of the issues, I will send that)

I think you are correct that this eventually needs to go to
problem-statement. Still reading.

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Thu, 19 Dec 2002 13:01:45 -0700

While I think its a good idea for chairs to probe about participation
via the mailing list and to determine who is paying attention at all
to the goings on, the membership idea seems to involve a lot of
unnecessary overhead.  It's a hasslement to keep a membership list
up-to-date; people come and go, their attention wavers, their time is
not always their own.  Would non-performing members be formally
removed?  Some people may be discouraged from even trying to
participate if they must tell their employers that by joining a WG
they have made a formal time committment; most supervisors would
flatly deny such a request.

I think that WG chairs are aware of their active participants, and in
my experience they do try to find out why an active participant has
gone silent.  I'd be happy if there were guidelines to chairs about
how many active participants there usually are in a healthy WG, what
to do if the number seems to be dropping too low (there are lots of
things), how long to allow a group to go on with a substandard level
of participation.

Hilarie

---

From: Marshall Rose <[email-masked]>
Date: Thu, 19 Dec 2002 15:30:09 -0800

hi. let's say i like the direction you're going, but i'd prefer to take
a step back first to make sure we're all on the same page:
    
i think that there's a lot of things that go into making a successful
working group. from a personnel perspective, i think that in addition to
management (the chair/co-chairs, technical advisors), you have three
tiers of membership:
    
    - a design team, usually <= 5 people
    - contributors, usually <= 20 people
    - the masses, usually < the earth's population
    
although there's some talk about design teams in various documents,
there's still something of an adhoc feeling about them, and contributors
don't have any mention at all.
    
now, i'll claim that all three of these tiers provide are necessary to a
successful outcome.
    
    - design teams provide focus and consistency
    - contributors provide breadth
    - masses provide buy-in
    
(this is a little rough, i'll come up with crisper sound-bites later.)
    
quick question: can we agree on the taxonomy?
    
if so, then sticking with fred's "management by objective" (or, more
plainly, "it's the charter, stupid!"), does it make sense to make design
teams and contributors an explict part of the chartering progress?
    
if so, can we also agree that, from time to time, charters get updated
as our understanding begins to converge with reality?
    
at this point, my claim is that we need to focus on two things:
    
    - what are the membership requirements for design teams and
      contributors?
    
    - how does formalizing the design team and contributor concepts
      impact the way the wg process works?
    
again, i like the direction you're going in (even if i'm not on board
with all the details), but i'd like to see if we agree on the
architectural aspects before we do the actual engineering.
    
ok?
    
/mtr

---

From: Ted Hardie <[email-masked]>
Date: Thu, 19 Dec 2002 16:06:30 -0800 (PST)

Hi,
	We're not all on the same page.  I think you're very far down
the path of a specific constellation of committed individuals,
when I'm still trying to get us to agree :

1) that not having any formal idea of membership has meant it is hard
to know who is committing to meet the milestones of a working group
and therefor it is hard to manage to those milestones.

2) that having a set of folks who have committed to doing the work
of the charter will improve the ability to manage to those milestones.

3) if "doing the work" is defined in terms of making definite
statements about the proposals, having a committed set of folks
"doing the work" may help us assess consensus by making review
of the statements a step in the process of that assessment.

3) we need to retain as much as possible the idea that anyone can
raise a technical issue at any time, so we need to ensure that
you do not need to have made this committment to raise an issue
as well as ensuring that anyone may make this commitment at
any time.

	So, before we jump ahead to your questions, can we back up to
these points and discover whether we agree here?
	If we don't, we need to go back to this stage of the
discussion.  If we do, I think I disagree with both the implication in
your note that a specific configuration of committed folk is required.
I've seen successful work where there was a "design team" of one, and
those in which multiple teams cracked small bits of a puzzle and the
group as a whole glued them together.
	I'm also concerned that making participation as a contributor
or a member of a design team part of the charter (or re-charter)
process significantly risks openness.  That's a mighty fine baby in
that bathwater, and I don't want to throw her out.
					Ted

---

From: Marshall Rose <[email-masked]>
Date: Thu, 19 Dec 2002 16:48:44 -0800

> when I'm still trying to get us to agree :

ok. if i read your note correctly, you're asking for other folks to
    
> these points and discover whether we agree here?
    
i agree. what do others think?

> group as a whole glued them together.

hmmm... interesting alternative. i agree that we should support this as
a part of the "architecture".

> that bathwater, and I don't want to throw her out.

yes, this is an important issue, which is why i ended my reply to you

/mtr

---

From: Andy Bierman <[email-masked]>
Date: Thu, 19 Dec 2002 17:29:19 -0800

>> and therefor it is hard to manage to those milestones.

depends on the size and newness of the WG.  It's easy for the Chair
of a small WG that's been around a long time to tell who is going
to be positive contributors.

>> of the charter will improve the ability to manage to those milestones.

yes, but I don't want to make the WG process too formal.  Well-intentioned
people may start out interested but not stay in the game, for various
reasons.  

>> of the statements a step in the process of that assessment.

I don't like the formalism of designated comment providers
in the WG.  If some document dies on the vine because nobody
is willing to take the time to read it and polish it, then
maybe that's a feature.  PACT's 18 month rule will filter
out this sort of draft.

>> as well as ensuring that anyone may make this commitment at

of course. Otherwise why bother with a WG process at all.

>> >     - what are the membership requirements for design teams and

1) participation in the IETF for at least a year
2) agreement (or no objection) from other design team members 
   and the WG Chair that the person is clueful enough to be on
   the design team
3) agreement from the candidate that they will commit enough
   hours to meet the schedule imposed on the design team by the
   WG Chair (based on charter milestones).  This includes attendance
   at interim meetings and conference calls, not just reading drafts.

>> >       impact the way the wg process works?

1) I think the WG Chair should have veto power over design team
   membership for their WG.  
2) A detailed mini-charter and schedule should be created and followed
3) Minutes from design team minutes should be posted to the 
   WG mailing list
4) Design team direction should be subject to WG comment. The WG
   should be able to comment on the design team effort at multiple
   points in the process, not just the outcome of the design team.

>/mtr

Andy

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 20 Dec 2002 12:35:23 -0800

Marshall Rose wrote:
> > Hi,
> > 	We're not all on the same page.  I think you're very 
> far down the 
> > path of a specific constellation of committed individuals, when I'm 
> > still trying to get us to agree :
> 
> ok. if i read your note correctly, you're asking for other 
> folks to state opinions on the four questions below:
>     
> > 2) that having a set of folks who have committed to doing 
> the work of 
> > the charter will improve the ability to manage to those milestones.
> > 
> > statements about the proposals, having a committed set of 
> folks "doing 
> > the work" may help us assess consensus by making review of the 
> > statements a step in the process of that assessment.
> > 
> > raise a technical issue at any time, so we need to ensure 
> that you do 
> > not need to have made this committment to raise an issue as well as 
> > ensuring that anyone may make this commitment at any time.
> > 
> > 	So, before we jump ahead to your questions, can we back 
> up to these 
> > points and discover whether we agree here?
>     

I agree in concept, but (2) is insufficient, and I am not sure how to
fix it. There have been cases like mIPv6 where a committed team put
substantial effort into aligning the document with wg consensus, but
late in the process a significant technical obstacle was raised. This
makes the hard milestone approach problematic, because even if the wg is
on track to meet the dates, it can end up getting shut down over a
detail that it missed, but is perfectly capable of correcting with
enough time. I picked that one because I was not directly involved in
document development, or even closely watching it, but I ended up
raising the technical objection that threw them off track. They thought
they were doing the right thing, and probably would have made it through
IESG review. The problem wasn't that the technical approach was wrong,
just that it was operationally unrealistic.

>  ...

I really have a problem with trying to define this too tightly. For the
design teams that ngtrans spun up, the leaders were specifically asked
to find people that were not long time IETF participants. We did this to
inject a customer perspective into what had become an increasingly
parochial view of the part of the problem space that developers were
willing to take on. If we require prior participation, we will end up
cutting off the viewpoint of how people want to use the technology we
define. The danger is we will end up basing future decisions on how a
prior technology was designed to be used, not how it is really deployed
in practice.

> > >       impact the way the wg process works?

IMHO, it shouldn't. What it would do is make it easier for new
participants to understand how the process actually works. Even in the
case of an individual submission, the design team = 1, contributors = 0,
and the masses = IETF last call. I think this is less about formalism,
than simply describing how we work.

Tony

---

From: Fred Baker <[email-masked]>
Date: Fri, 20 Dec 2002 11:24:35 -0800

>don't have any mention at all.

??

I tend to think the first are the authors, and the second go into the
acknowledgements. Both of those are fairly clearly spelled out, I should
think.

But yes, I think we are on the same page for the taxonomy.