Subject: Avoiding last minute issues

From: [email-masked]
Date: Mon, 2 Dec 2002 08:58:51 -0500

> > participate, and provided, at best, some sort of Area membership review
> > and eliminated IETF wide review, etc.,
> 
> IEEE *does* have wide review -- it just doesn't find a large number of
> critical last minute issues in that process.

I think early review by experts from other areas would help a lot
with avoiding last minute issues.  As I said at the plenary, it's
not just security, but also other issues like naming that can be
caught this way (ditto appropriate use of DNS, HTTP, etc.).  I've
been on the other side of this as an expert helping a WG to get an
addition of diffserv support to a protocol right, and I'm currently
involved with some transport issues in IPsec (specifically IKEv2).

To the extent this sort of thing happens today, it's informal and
haphazard at best (with the notable exception of the MIB doctors)
- it requires a clueful WG chair to realize that <X> is going to
be a problem and actively go look for help with <X>, or the
concerned expert to invest the time to make sure that the WG
gets <X> right even though the folks involved may not realize that
<X> is going to be a problem.  Writing down some of the things
that "everyone knows" about what the likely problem areas are
and how/where to go get early help with them would help (e.g., the
security area advisors, and whatever resources other areas choose
to set up) - some of the areas are in the ID-nits document, but
it's not a complete list, it contains a whole bunch of other stuff,
and it often isn't consulted until late in the process when the
authors think that their draft is close to done.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953 **NEW**     FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2002 06:51:00 -0800 (PST)

> Writing down some of the things
> and how/where to go get early help with them would help

During IETF 55, we discussed the idea of a "100 problems" document.
This would be a compendium of the 100 most common major problems
encountered in documents, and some suggestions (or references) for getting
around them.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 02 Dec 2002 10:56:17 -0500

> concerned expert to invest the time 

I think this is actually the main issue here - time. Most of the 
participants at IETF that I know are very busy people, with full-time+ 
regular day jobs in addition to their IETF work. What little time 
remains for IETF activity is usually spent on the things they care about 
  - the documents they are editing or the groups that affect their work. 
That leaves very few people and very little time for further volunteer 
work to review other people's documents.

Ultimately, I believe that is the root cause of many of the problems 
related to document quality before reaching IESG. Someone on this list 
(I forget who posted it) desribed their view of the typical way 
documents progress - someone writes a doc, it receives few comments, the 
group says "yes" to it, the author eventually does an update or two, and 
the group throws it to IESG with little group input. I also agree that 
this is a common mode of operation. It is symptomatic of a process where 
people are incented to care about their own work, and pay little 
attention to other documents that are peripheral to their primary 
interest. One might argue that its symptomatic of groups taking on work 
that it shouldn't; if so few people care, the work should never have 
been adopted. In the end its probably a combination of the two.

However, I believe that ultimately, the process needs to evolve to make 
a larger group of people feel accountable for the quality of the 
documents that reach IESG. Accountability is the key to any management 
process.

How can we make groups feel more accountable for document quality and 
timeliness? Some random ideas:

1. For each document that reaches IESG and gets "bounced" because of 
some problem, the group will see an additional delay of X weeks 
explicitly added to IESG consideration of any other group document, at 
the discretion of the IESG.

This helps incent working group chairs to forward quality documents to IESG.

2. For each document that reaches IESG before its chartered milestone, 
that document goes ahead of others in IESG consideration and RFC editors 
queue. As a result, really late documents end up being even later.

This helps incent working group chairs to forward documents on time to 
the IESG.

3. Every document editor commits to do two reviews of other documents, 
chosen by the IESG and/or working group chairs. Those reviews must be 
delivered on time based on a schedule negotiated with the chairs. If a 
review is missed, the document being edited is either explicitly delayed 
by IESG, or, the editor is replaced.

THis helps incent editors to review other documents.

The SIP working group has tried a variation on item three, whereby 
people that came to ad-hoc gatherings on a certain topic during the IETF 
committed themselves to reviews. However, lack of mechanisms to ensure 
accountability for those reviews resulted in limited success.

Now, I am not saying that we really should do any or all of the above. 
My main point is that unless the process has an explicit means to make 
people feel truly accountable for certain deliverables, we will always 
have problems. In the real world, such accountability occurs through 
salaries and bonuses. In a volunteer organization, other things are needed.

Thanks,
Jonathan R.

-- 
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: Marshall Rose <[email-masked]>
Date: Mon, 2 Dec 2002 09:36:29 -0800

folks - is there a reason why we aren't discussing this stuff on the
problem-statement mailing list? is there something wgchair-specific going on
here?

/mtr