Subject: RE: Milestones

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Thu, 12 Dec 2002 01:03:03 +0100

> -----Original Message-----
> From: Fred Baker [mailto:[email-masked]]
> Sent: woensdag 11 december 2002 22:07
> To: Wijnen, Bert (Bert)
> Cc: [email-masked]
> Subject: Re: Milestones
>
>
> At 02:20 PM 12/10/2002 +0100, Wijnen, Bert (Bert) wrote:
> > > PLEASE NOTE:  MOST PEOPLE IN THE IETF DO NOT TAKE THE
> >                 MILESTONES SERIOUSLY.
> >
> >So why do we (you) as WG chairs not change that ?
>
> because the ADs don't enforce them, and neither the ADs nor
> the Secretariat pings anybody about them.

Well, I regularly ask my WG chairs to update their milestones.
Some do, others don't...

> When was the last time a working group was told "you have
> no future milestones, so you can't meet"?
>
I have not done that. We're having the discussion if we should
indeed use the milstones more forcibly to kill/shutdown WGs
or to remove work items from their charter. That is a good
discussion to have, and once we all agree, then we can enforece
it more easily I think.
AS far as I can tell, all the time I have been an AD, the
milestones have never been used to kill/close a WG if they
missed their milestones even by years. So I think out culture
is not used to it. If we all want it, we can change that,

Why I think milestones may still be usefull (even if they get
pushed out frequently) is that if a WG member, and IESG member
or a newcomer or outsider, looks at a WG charter page, they
get an impression, an idea as to where the WG is working on
and what their current thinking is about when they want/think
to finish. I think that is good info to have.
It should have realistic dates in that the WG and the editors
believe they can make the dates.

Bert

---

From: Fred Baker <[email-masked]>
Date: Wed, 11 Dec 2002 17:29:05 -0800

>is not used to it. If we all want it, we can change that,

In point of fact, MTR did so regularly during his tenure as NM AD. It got
people pretty upset at times, but it was in fact effective. A lot of work
got done on something resembling a schedule. Ditto the editing of RFC 1812;
I was given a certain time period, and there was a clear expectation that
it would get done in that timeframe. My day-job management basically told
me to take whatever time I needed and do so.

This has been done in the past, and can be done now.

So I have a modest proposal.

(a) I would like to see charters changed slightly, to show the initial date
for a milestone and the current expected date, and showing "done" as a
separate checkoff with an "actual" date instead of an "expected" date.

(b) I'm willing to work through whatever process to update them. Initially
that sounds like email to ietf-action sent either by the WG Chair or the
AD. But it does sound like an AD or WG Chair should be able to update the
charter more-or-less directly - for example, the WG Chair could have access
to a form that lets him update the milestone dates, and gets mailed to the
AD. When the AD forwards it to <the right place>, the new information is
substituted for the current expected date.

(c) The secretariat should be given summary permission to give a working
group with future charter milestones a meeting or two at an IETF, with or
without AD notification. A working group without future milestones (eg,
milestones with future expected dates) should be granted such only with AD
concurrence.

(d) the secretariat should ping the working group chair, copying the
relevant AD, as milestone dates are crossed: "on today's date, the
<example> working group is supposed to submit the <whatever> draft. Have
you done that? Please update your charter, either checking it off as 'done'
or showing a new 'expected date'." Of course, milestones that are done and
checked off need not be pinged about.

You have heard this from me before. I think doing this (management by
objectives) will very likely reduce the level of concern in the IETF, as is
being discussed on the "problem statement" list.

The other obvious change is that whenever an AD makes a statement to a
working group representative, such as "I have reviewed draft <mumble> and
it needs <this> done to it before I can advance it with a clear
conscience", the working group needs to be copied on the email. It might be
nice if it was also posted somewhere for ease of finding it ("what is the
status of <draft>? Oh, the AD sent it back for stated updates..."), but
that is a secondary issue if working group mailing lists have archives.

Response requested: Working group chairs that feel that this would be a
wise change to our procedures, please say so. People that would like to
*not* do this, please say so, say why it's a bad idea, and say what you
think would be a better solution.

---

From: Paul Hoffman / VPNC <[email-masked]>
Date: Thu, 12 Dec 2002 10:04:41 -0800

Sounds good to me. Your proposal:
- lets IETF members know what is really happening
- puts the onus on WG chairs to meet deadlines and/or to explain why
- puts the onus on the ADs to manage the WG chairs, not the WG output

--Paul Hoffman, Director
--VPN Consortium

---

From: Randy Bush <[email-masked]>
Date: Thu, 12 Dec 2002 10:09:08 -0800

> - puts the onus on the ADs to manage the WG chairs, not the WG output

so, when the output is mediocre..bad, we keep firing chairs until the
quality is reasonable?  aside from other non-benefits, i suspect this
is not the best way to create timeliness.

randy

---

From: Paul Hoffman / VPNC <[email-masked]>
Date: Thu, 12 Dec 2002 10:19:48 -0800

At 10:09 AM -0800 12/12/02, Randy Bush wrote:
>quality is reasonable?

Did I say that? Do you feel that the only way to manage someone is to 
fire them? How about:
- Suggesting how they might re-charter to narrow the focus
- Suggesting to them how to get more effective document editors
- Suggesting to the WG that they should listen to the chairs more
. . .

>i suspect this

Agree. That's why I didn't suggest it.

--Paul Hoffman, Director
--VPN Consortium

---

From: Randy Bush <[email-masked]>
Date: Thu, 12 Dec 2002 10:29:14 -0800

> - Suggesting how they might re-charter to narrow the focus

funny, i am in the middle of a conf call on this very thing right
now.  the pushback to keep a boil the ocean charter is amazing.
having been unable to communicate that this is a serious problem
(which has been mentioned by others as well) i have to decide
whether to let the mess go and deal with the mess it makes over the
next year or two, stall and talk while trying to get through the
"the wg has consensus on this and you mean ad should not be
blocking this", fire, or what.

> - Suggesting to the WG that they should listen to the chairs more

that last is clearly an AD job.  back up your people.  but i don't
buy acting for them.

randy

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 12 Dec 2002 09:24:01 -0800 (PST)

> - Suggesting to the WG that they should listen to the chairs more

None of these things provide incentives for the WGs to apply more thought
to the problem. When WG work items are *really* late, it's often because
some aspect of the problem hasn't been understood. Yes, there can be
distractions, bad management, etc. -- but ultimately WGs move at the speed
of thought.

Often the "hard problems" that need thought can be understood early on --
but we don't typically think of WGs as enterprises that require a complete
team, covering all the bases, in order to succeed.

So to my mind, missing milestones is a symptom, not a cause.

---

From: "James Kempf" <[email-masked]>
Date: Thu, 12 Dec 2002 11:28:29 -0800

> think would be a better solution.

As long as the milestone does not reflect a dependency on another action
happening, I think this could work. For example, a milestone that says "Document
to RFC" is dependent on the IESG completing review and approving, and the RFC
editor process completing. These are out of the WGs control. The milestone
should say "Document to IESG for review". That is something that is completely
in the WGs control. If it is within the WGs control, then holding the WG chairs
and WG to it in a fashion as you describe above is entirely reasonable, IMHO.

            jak

---

From: "James Kempf" <[email-masked]>
Date: Thu, 12 Dec 2002 11:40:55 -0800

Randy,

> is not the best way to create timeliness.

This proposal is talking about timeliness w.r.t. goals set by the WG themselves,
not w.r.t. assessment of quality at the very end by the IESG, as I understand
it. If an AD that is monitoring the progress of a group finds a problem with
quality, and, after discussion with the IESG, decides that the problem is due to
the WG chair, the chair should be removed IMHO. If the problem is due to the
dynamic of the WG (wrong people involved, people involved are clueless, etc.),
then the WG should be dissolved and the community of interest told to re-BOF, or
the AD should find someone with clue who is willing to inject themselves into
the process and wrestle with the WG to improve the quality. Finding a quality
problem at the very end, during IESG review, is perhaps the worst possible case,
because it comes latest, and it is hard for the IESG to really make a difference
in the outcome without looking as if it is holding up timeliness.

So, yes, if there is a quality problem and the WG or WG chair look to be the
problem, then the WG or WG chair should be fired until it improves, with early
action preferable to later. In addition, holding WGs to their deadlines is a way
to achive timeliness for WGs with good chairs that are progressing with good
quality, but are being held up by nit-picking or inattention.

            jak

---

From: Fred Baker <[email-masked]>
Date: Thu, 12 Dec 2002 11:17:23 -0800

At 10:09 AM 12/12/2002 -0800, Randy Bush wrote:
>when the output is mediocre..bad, we keep firing chairs until the
>quality is reasonable?

As Bernard says, missing milestones is a symptom. But it is a measurable
one. When your kid has a fever, you go looking for why, and you do what you
can to fix it. The issue is not usually the fever, it is something else,
and you use the fever as an alert and a measurement tool.

If changing the chair is the right fix, that's what you do; if changing the
contributors is the right answer, you do that. If the problem is that the
working group's problem statement is inadequate, that's what you go after.
If the working group is moribund or off in the weeds, you deliver the coup
de grace.

This is pretty much "standard management practice", as I understand the term.

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 12 Dec 2002 15:15:31 -0500

Hi Fred,

I think that your proposal has several good points, but that
may be because I am naturally inclined towards the management
by objective approach.  However, different ADs  have very
different management styles, and not all of them may be
comfortable with the MBO approach...

>separate checkoff with an "actual" date instead of an "expected" date.

This might be useful, but what makes you think it will be maintained
properly, when we can't seem to maintain the dates that we have now?

>substituted for the current expected date.

How is this simpler than what I do now:

         - Cut and past current milestones into a mail buffer
         - Modify as appropriate
         - Mail to ietf-action and cc: the AD

>milestones with future expected dates) should be granted such only with AD

Do we have any reason to believe that AD approval of meeting time
is currently causing a bottleneck of any kind?

>checked off need not be pinged about.

If this could be automated, so that it didn't require any human
secretariat time, it would be helpful, I think.  Otherwise, it
probably wouldn't be worth it.

I doubt that I would consider this the highest priority project
for automating the process, though, so it would have to be
weighed against other possible projects.

>being discussed on the "problem statement" list.

I actually agree...  MBO is a good system for creating a "contract"
between a second-level manager and a line manager and gives the line
manager discretion to act within the contract.  And I personally
feel that more discretion, responsibility and accountability at the
WG chair level would improve the scaling properties of the IETF and
help us get more work done...

>conscience", the working group needs to be copied on the email.

True.  In general, we need to get more of this discussion out in the
open.  I think we've made a big step in this direction this year with
the I-D tracker, and I hope we'll keep moving in that direction.

>think would be a better solution.

I would like to see some tracking dates (original date, revised date,
completion date) on the charter pages to enable an MBO approach.  I
would also like to see my ADs use this type of approach, but I don't
think that it should be forced on them.

I would not like to see us spend our scarce automation resources on
a milestone reminder system or a milestone update system, as I think
there are higher priority needs, like adding more features to the
I-D Tracker.

And, unless it is a big drain, ADs should probably continue to
approve meeting slots.

Margaret

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 12 Dec 2002 15:28:01 -0500

>the WG chair, the chair should be removed IMHO.

What is this fixation with removing WG chairs (even I have it sometimes).
What about communicating the problem to the WG chair and educating them
so that they won't make the same mistake again?  Maybe this education
could happen before documents go to the IESG, if IESG or IAB members
notice a problem.  Or, if the problem isn't noticed until the document
is in the IESG, work with the WG chair to figure out how to correct
it.

We work in an environment where it is culturally discouraged to show
any sort of weakness, and where people _enjoy_ being so cryptic
that no one knows what they are talking about...  Maybe we need to
change that.

Let me give an example from my own experience:  Lots of folks have
made snide things about the tendency of my v6ops working group to
build transition mechanisms that rely on relays...  So, what's
wrong with relays?  I honestly don't know, and I don't even know
how to find out.  Should I be fired for this fact?  Perhaps.  Or,
perhaps someone could just explain it to me...? (Although it may
not be best to do it on the wgchairs list.)

None of us were born perfect, and I'm pretty sure that none of us
achieved perfection before becoming WG chairs.  As long as we are
trying to do the right thing, and show a willingness and ability to
learn to do our jobs better, what good would it do to continually
replace us with new, equally imperfect people?

>If the problem is due to the
>the process and wrestle with the WG to improve the quality.

There are two possible cases here:

         (1) The work the group is doing is fundamentally flawed and
                 not important.  In this case, flushing the group
                 is probably the right thing, but there is no need
                 to re-BOF.

         (2) The work is necessary, but the group isn't executing
                 as well as the AD would like.  In this case, the
                 group needs to be re-focused, possibly re-chartered,
                 and educated regarding how to do the right 
thing.
                 Some new people should probably be recruited to inject
                 clue, etc.

                 This is an extremely painful and lengthy process, but
                 it is the only way to get to the desired goal.
                 Running another BOF with the same people won't help.

Margaret

---

From: Fred Baker <[email-masked]>
Date: Thu, 12 Dec 2002 12:17:24 -0800

At 11:28 AM 12/12/2002 -0800, James Kempf wrote:
>For example, a milestone that says "Document to RFC" is dependent on the
>IESG completing review and approving, and the RFC editor process
>completing. These are out of the WGs control. The milestone should say
>"Document to IESG for review".

yes; this is called "write the charter correctly". That said, it's not
quite true that the WG is unable to affect the progress of the IESG's
review; documents are often sent back for update and the updates take a
long time, but the official status is that the AD is reviewing the
document. I can tell you of any number of cases that have gone roughly like
this:

     The working group sends a document to the AD, who says "I won't accept
this
     because you haven't got", to pick one common problem, "an adequate
security
     solution." The WG chairs and authors are told to come back with a viable
     solution. This is not copied to the list (mumble) because tradition
says it
     goes to the chairs and the authors. The authors fail to respond, and
     complain that the IESG is unreasonable, and the working group gets
upset at
     "IESG inaction", while the AD patiently awaits a credible document.

You would be amazed at how many document reviews fall into some variation
on that scenario. That, specifically, is why I would like to see such
comments copied to the WG list, where they are archived and visible.

A specific scenario that happened with the MPLS drafts is similar, but
related to the RFC Editor. The drafts they sent at one point all had a
normative reference to some particular draft, and that draft stayed in the
working group for over a year after the IESG had passed the remainder on. I
periodically got pretty stiff notes about "IESG inaction" from working
group members (and from Cisco's marketing department), and found myself
forwarding copies of emails pointing to the document statuses. The IESG
took the fall for something it had no involvement in: the RFC Editor was
patiently waiting for the working group to get a key document out.

---

From: Mukesh Gupta <[email-masked]>
Date: Sat, 14 Dec 2002 08:12:19 -0800

Though I am new to all this, here is my opinion. I don't agree with some people
who said that the milestones change faster than they can be updated. I
rencently updated the milestones for my WG and the process just took around 2
weeks. The milestones can at least, if not more, be updated after every IETF
meeting. Margaret (and I am sure there will be more like her) does update them
after every meeting. If she can do it why can't others. Having a clear set of
goals and milestones is absolutely necessary in order to maintain the focus.

> separate checkoff with an "actual" date instead of an "expected" date.

Showing the date the milestone was completed is a very good idea. I don't
oppose showing other dates too. Another date that would be good to show is the
date when the milestones were last updated on. This will help people figure out
how updated the milestones are. The secretariat can send a friendly reminder to
the chair if this date is one year old or so.

> substituted for the current expected date.

I don't know about people's experience about this but I have just updated my WG
milestones and it was such an easy process. I had a hallway chat with my AD in
Atlanta about the milestones. I sent them for review to the WG and when
everyone agreed, I sent a mail to ietf-action copying my ADs. Within two days,
the milestones were updated. Though, even I would prefer having a better
mechanism like the one you are suggesting, I agree with Margaret that the
current process is not as bad as people think.

> milestones with future expected dates) should be granted such only with AD

I agree.

> checked off need not be pinged about.

I agree.

> that is a secondary issue if working group mailing lists have archives.

I agree.

regards
Mukesh

---

From: Marshall Rose <[email-masked]>
Date: Wed, 18 Dec 2002 22:41:43 -0800

> ...

fred - sorry for the delay in responding.

fundamentally, i have to agree with you. (and not the least of which
because you're citing me!)

philosophically, i think that charters are best understood as contracts
between the wg and the ietf, which the iesg negotiates and enforces on
behalf of the ietf. and your "management by objective" proposal is
certainly a consistent, and rather forceful, implementation of that
philosophy.

that the modern iesg has never felt "empowered" (for lack of a better
term) to undertake that management philosophy is, as Bert suggests,
something we can move towards.  the issue, of course, is whether the
ietf has the will to do so.

and, in a real sense, this is key issue that the iesg, the wgchairs, and
the "key contributors" have to face: are we willing to go back to a more
rigorous style of management where milestones are *actually* enforced?
i hope that answer is yes, because most of the other proposed solutions
probably introduce a lot more pain into our lives.

now, as one of the co-authors of the PACT document, some might think it
odd that i'm endorsing "management by objective". the reason, of course,
is that the PACT document tries to get the same result, but as a
side-effect, i.e., the PACT document says each working group gets 18
months to prove itself (where "proof" is demonstratrated by getting an
I-D approved by the iesg).

in this matter, the idea behind the 18 months is simple: rather than
have the ADs audit the milestones frequently, it's easier to set one
"meta-milestone" that's fatal if you miss it.  the idea, of course, is
that each wg, being composed of rationale actors, will organize itself
to meet the meta-milestone, and, as a side-effect of this, the regular
milestones will get met.

although "management by objective" requires regular AD interaction, i
don't view this necessarily as a drawback, and, obviously, i like the
fact that good things happen directly, rather than by side-effect.

in fact, if i look at the summary of proposed rules in the PACT
document, then i can easily substitute fred's probably as replacing rule
2 (the "18months to prove yourself" rule) and augment rule 1 (the
"charters must be clear" rule).

/mtr

---

From: Fred Baker <[email-masked]>
Date: Thu, 19 Dec 2002 00:07:27 -0800

At 10:41 PM 12/18/2002 -0800, Marshall Rose wrote:
>i can easily substitute fred's probably as replacing rule

works for me.