Subject: Issues summary from the discussion

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 07 Nov 2002 11:19:36 +0100

at last, a summary.....
important issues I missed?
issues that people wish to disavow?

sorting & prioritizing will take some time....

format apology - if someone knows the magic incantation to make links -dump 
produce 70-character-wide output, please email me!

             Harald
---------------------------------------------------------------------
                      Issues/Problems in the IETF process

   version 0.1 - http://www.alvestrand.no/ietf/issues.html

   This list came about by reading a discussion on the wgchairs mailing 
list,
   October/November 2002.

   The thread was started by a call for volunteers in helping draft a 
problem
   statement, but quickly turned into a discussion of what the problems 
were.

   In the list below, I have tried to succintly reduce each issue to one
   sentence - since I'm doing duplicate suppression, many people will not 
see
   THEIR name in the list, but there are undoubtedly many who would add 
their
   names to the statements if they had the chance.

   The issues have not been sorted or grouped - I thought it better to get
   the list out now than to wait another day to sort it.

The problems

   IETF process is opaque, and therefore hard to coordinate with (Dean
   Willis)

   Problems (for instance security) get discovered at IESG time, not earlier
   (Randy Bush)

   Not enough guru bandwidth for appropriate security review (Derek Atkins,
   Steve Bellovin)

   Not enough information for the WG chairs or authors to identify the IESG
   concern and effect the resolution in a timely manner (Dean Willis)

   WGs are not dilligent enough in ensuring quality control (Randy Bush)

   Predictability, Accountability, Competency and Timeliness are important 
to
   the IETF, and we don't have enough of them (draft by Marshall Rose, Geoff
   Huston)

   Time from WG finished to IESG approval is too long (Frank Kastenholz)

   We don't have well-defined success criteria, and don't measure the ones 
we
   could have (Bernard Aboba)

   Mailing lists are not (always) an effective tool (Aaron Falk)

   Individuals' behaviour has changed (less commitment)? (Aaron Falk)

   Less consensus on what parts of the IETF's work is "core" and "critical"
   (Aaron Falk)

   poor quality documents tend to act as DoS attacks... sucking away
   process/review bandwidth from quality documents. (Kurt Zeilenga)

   Too little clarity and brevity in WG output (Aaron Falk)

   At the moment, we collectively accept almost any working group, and allow
   them to produce almost any result. (Joel Halpern)

   Trying to measure the progress of the IETF doesn't gather enough interest
   to be worth it (Marshall Rose)

   For any X in the IETF process (WG, IESG, RFC Editor), it takes too long
   (David Meyer)

   Consensus process is not working, or is too slow, due to lack of interest
   in the common good (Melinda Shore)

   Large groups don't interact well enough to form consensus (James Kempf)

   WG process is too slow, because of feeping creaturism, deadlocked
   conflicts or lack of worker bandwidth (Jari Arkko)

   Lack of running code means that unimplementable specifications get on
   track (Kurt Zeilenga)

   Lack of transparency makes it hard to diagnose what the organization's
   problems are (Hilarie Ormond)

   Large WGs means that we get more groupings of intrasingent people, 
leading
   to difficulty with consensu  (Donald Eastlake)

   Authors/editors do not understand what makes a document Good; they should
   be trained (Avri Doria)

   Few people review drafts in IETF Last Call (Avri Doria)

   There are a number of issues surrounding MIB modules (Bert Wijnen)

   Queueing delays inside the IETF process are far too long (Don Eastlake)
   (suggests that 1-2 months is most often a non-problematic delay, but that
   1 year is a disaster)

   What consensus means has been lost, or the spirit of consensus is 
missing
   (Edward Lewis)

   Discussions in WGs turn into battles along "party lines" between
   companies, not an effort towards interoperability (Edward Lewis)

   Heavy investment in alternatives pre-standardization leads to vested
   interest in outcomes, preventing consensus when there is no clear
   technical lead (Frank Kastenholz)

   Ratholing over fundamental disagreements on the goals of the process 
leads
   to bog-down; requirements documents can sometimes help avoid this (Tony
   Hain)

   We're not using the three-stage standards process as designed (Spencer
   Dawkins)

   I-Ds that are of the "wouldn't it be a good idea if this worked" type
   confuse the process (Eric Burger)

   The process' slowness means that I-Ds take on the force of a standard
   (Eric Burger)

   We as a community are not following the rules we set for ourselves (James
   Luciani)

   we think there are cases where things go to Proposed Standard specifying
   things that can't work.
   One way to avoid that is to insist that someone try it before submission.
   (Harald Alvestrand, summarizing)

   Architectural decisions with IETF-wide impact doesn't get adequate
   community review (Margaret Wasserman)

   It's not possible to identify the time when major changes to an I-D set
   are unlikely, so that it's reasonably safe to start designing
   implementations (James Luciani)

   It's hard to get a discussion going when the IAB works on architectural
   considerations (Rohan Mahy)

   Documents that bypass the WG process sometimes have huge impact on the
   work we have to do (Glenn Parsons)

   Some consider IAB architectural guidelines/question to be "meddling" by
   "outsiders" (Melinda Shore)

   Insularity is unavoidable given the large number of working groups
   (Melinda Shore)

   There are no roadmaps for areas or other large chunks of the architecure,
   just for WG efforts (David Harrington and others)

   Mailing lists with large numbers of postings (500 in a week, for 
instance)
   are hard for peole to follow (Thomas Narten)

   Too little architectural guidance in the IETF (Carl-Uno Manros)

   It is impossible to define a single architecture for the Internet (Brian
   Carpenter)

   People sticking to fixed positions on mailing lists leads to lots of mail
   but no consensus (James Kempf)

   Maintaining decorum and forward progress on a mailing list is not easy
   (many)

---

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 08 Nov 2002 02:10:48 -0500

Another problem I'd like to point out relates to some of the points made 
by Avri on training of document editors.

I think many editors/authors are not sufficiently familiar with the set 
of "core" RFCs that document much of the IETF wisdom that generally 
applies to protocols. For example, we have a standard format for BNF in 
protocols (RFC 2234), however many protocols subsequent to it defined 
their own BNF, or variants from RFC 2234. There are pitfalls to that, of 
course (implicit vs. explicit LWS), and if you follow RFC 2234 you avoid 
some of them.

Perhaps some of the quality issues would be avoided if document editors 
were aware of, and had read, these specifications.

I would find it terribly valuable to have a "must read" list of 
specifications for all document editors. These would include things like 
RFC 2234, RFC 2434 (IANA guidelines), RFC 2396 (URI syntax), Eric 
Rescorla's excellent I-D on writing security considerations, documents 
on internationalization issues (something I see several working groups 
simultaneously addressing in different ways), obvious ones like RFC 
2026, and so on. I suspect there are a lot of really handy RFCs out that 
folks don't know about, which would provide useful guidance on avoiding 
common problems.

-Jonathan R.

Harald Tveit Alvestrand wrote:
> format apology - if someone knows the magic incantation to make links 
> -dump produce 70-character-wide output, please email me!
> 
>             Harald

-- 
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: "W. Mark Townsley" <[email-masked]>
Date: Fri, 08 Nov 2002 10:37:59 +0100

> folks don't know about, which would provide useful guidance on avoiding

Once the material you mention is put together into a handy package, perhaps we
could even have a training session at each IETF. ADs and WG chairs could use
attendance as deterrent to submitting bad specs... e.g. if your doc gets bounced
too many times for such style issues, one could be respectfully requested to
attend the training session (either in person, or at least via remote e-learning
of some sort) before resubmission.

Aside of whether or not one follows the RFC guidelines, there is an issue of
technical writing competence in general. I'm not so sure we can address this in
the IETF, aside of requesting one to sign up for a course at a local college
(1/2 kidding).

---

From: "W. Mark Townsley" <[email-masked]>
Date: Fri, 08 Nov 2002 10:37:59 +0100

---

From: Melinda Shore <[email-masked]>
Date: Tue, 12 Nov 2002 18:16:40 -0500

Although because this is a volunteer organization we need to
be sensitive to the need to avoid making participation
intolerable, if we're talking about additional training we
might want to consider providing information about
decision-making to those chairs who want it - what consensus
means, how it works in an ideal situation, what to do when
you hit a wall, how to ask questions that can be answered,
etc.  I'd be happy to help.

Melinda