Subject: RE: How long does it take on an average for WG I-Ds?

From: "Putzolu, David" <[email-masked]>
Date: Wed, 11 Jun 2003 13:04:48 -0700

I think it varies a lot.  In my case I'm going through a 
timeline that is actually quite similar to what you listed. 

I'm not sure what can be done about it though.  WGs can
take a while to get consensus, ADs need to consult with
other experts, IESG needs to deliberate - all of these
things take significant time.

-David

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> Sent: Wednesday, June 11, 2003 8:59 AM
> To: [email-masked]
> Subject: How long does it take on an average for WG I-Ds?
> 
> 
> 
> How long does it take for an I-D in the current system?
> 
> At least one view:
> 
> From the point of: Completion of I-D by editor or author of a WG draft
> (that is after it has undergone a few revs in the WG itself)
> 
> 1. WG Last call					  2 Weeks
> 2. I-D revision based on WG LC comments 	  ~8 Weeks
> At which point WG chairs submit the draft to the ADs for review
> 3. AD review					  ~6 weeks
> 4. Time for addressing AD comments and I-D revision ~8 Weeks
> Draft submitted to ADs for IETF last call
> 5. IETF Last Call				   2 Weeks
> Comments are expected from IETF LC. Rarely are any received.
> 6. IESG review					   ~8 Weeks
> 7. Time for addressing Discuss items and submitting
>    revised I-D			     		   ~12 weeks
> 8. Time to get IESG approval on revised I-D	   ~4 weeks
> 9. RFC ed queue					   ~12 weeks
> 
> Total time : About 62 weeks 
> 
> Is this the general experience of others in the IETF? Thoughts?
> 
> -Basavaraj
>

---

From: [email-masked]
Date: Wed, 11 Jun 2003 15:46:18 -0500

>On Wed, 11 Jun 2003 [email-masked] wrote:
>Why so long?  (The document should already reflect WG consensus pretty 
>well, and a lot of *major* issues shouldn't turn up now, much less need 2 
>months to resolve.) 

A few factors go into this. 
a. LC comments may result in lengthy discussions on the ML. This
   happens as a result of some WG folks who wake up only as a result
   of WG LC and have not actively been involved in the work.

b. Authors/Editors are not just doing work on IETF drafts. They also
   in many cases have daytime jobs and hence attending to the comments
   and resolving them to the satisfaction of the WG takes time. 

Well in an ideal case, there would be no or very few comments which
are resolved quickly and you have a very proactive editor who updates
the drafts without much delay. This does happen.... But is rare :(

>
>> 3. AD review					  ~6 weeks
>
>Why so long?  (If the spec was simple and of good quality, it would need 
>less attention from the AD, and he might be able to do it in 1-3
>weeks.)

From the time the WG chairs submit a draft to the AD and the AD finds
time to review and comment on it (note the busy schedules of the ADs),
it takes at least 4~6 weeks. I dont think the draft quality has
anything to do here. And you cannot always expect all drafts to be
simple or only 15 pages in length.

>Why so long? (Again, why so many major issues crept up which were 
>difficult to resolve?)

The reason being that the AD has not necessarily been paying attention
to all the discussion items in the WG. With the number of WGs and
drafts that an AD has to pay attention to, this is hard. Hence
invariably the AD raises comments and questions that may or may not
have been discussed earlier by the WG. But still it takes time to
respond to the AD and get agreement. There is latency in terms of
whether the editor/WG agree with AD comments and hence the email
ping-pongs take time and also the AD does not have just this one I-D
on his/her plate.

>Why so long? (See 3.)

Because thats how long it takes to get a draft on the IESG agenda and
get them to read and comment on it.

>Why so long? (See 4.)

See 4. 

>Ok.
>9 not really important as the RFC has already been approved.   In any 
>case, I recall the time being much less for actual editing job rather than 
>waiting for normative references to go RFC, or whatever.  But I don't 
>recall the details (some IESG/IAB plenary) and can't find them right
>now.

I agree. But still take the case of Diameter which for example was
approved by the IESG in December I think. Is there an RFC number that
I can refer to yet? NO.... If I am implementing the protocol and need
the IANA numbers and the RFC (just to ensure that there are no
surprises as a result of 48 hours note etc.), I wish the time taken
was smaller.

>
>What do I mean with "Why so long?"  Well, two factors:
>
> 1) if the document was of appropriate quality (in all aspects), there 
>would be fewer issues raised, and they would be easier to address
>

Agreed. But it still does not reduce the time by 50%, since some of
the delay is just a matter of procedure and availability of time for
ADs, IESG to review the document.

> 2) I really don't see how, on average, you need 2 months to edit a 
>document to resolve issues (unless the spec is very complex and gigantic; 
>also see 1)

It depends on the size of the document, availability of time by the
editor/author and ability to reach consensus on issues.

>
>But, of course, this is how it *should* be, not how it is in practice.. 
>:-)

Exactly :)

>
>-- 
>Pekka Savola                 "You each name yourselves king, yet the
>Netcore Oy                    kingdom bleeds."
>Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
>

-Basavaraj

---

From: "Harrington, David" <[email-masked]>
Date: Wed, 11 Jun 2003 17:21:25 -0400

Hi,

I think how long it takes depends on a lot of factors. Some are
controllable by the WG chair; some can be influenced by the WG chair,
and some are outside the control of the WG chair. 

I see lots of these emails complaining about the time it takes to get an
I-D published. But time after time, it is explained that sometimes the
delay is caused by the quality of the documents themselves.

I think an important question to ask yourself is, how well did I as WG
chair prepare the way for a short review/publication cycle?

1) If there were comments during the WG last call, were they
unanticipated comments? Or were they comments to fix things like typos
and rfc-nits, which could have been caught and corrected before the last
call, if somebody had been given the task of providing a through review
for such things. Did YOU provide a thorough review of the document for
typos and ID-nits before submission?

2) Did the WG write large documents that take a long time to review, or
was the work broken down into "manageable" chunks in different
documents? Area directors (and other reviewers) need to be able to
allocate time to do their reviews. If the document is sized and
modularized such that a thorough review can be done in two hours, the
reviewer will probably be able to fit that into their schedule fairly
easily; if the document is huge, and would require an allocated time
block of 8+ hours to do a technical and/or nits review, then it will be
much harder to fit that into a reviewer's schedule. As a mib doctor,
I've had to review some huge documents, and finding an available 8+ hour
block in my work week is very hard. Some of the mibs could have been
written in smaller document chunks, and I could more easily have found
four two-hour blocks to do my reviews. When you need to review a
document that takes 8+ hours, you typically cannot just do it in 2-hour
blocks because you must spend a lot of time jumping back and forth
within the document. When the document is broken into smaller documents,
where they can be reviewed in modules in dependency order, it is much
easier. WGs have refused to create smaller documents, despite requests
to break the documents down, and the huge size of the documents can
definitely delay the review process, by months!

3) Apparently the area director had a sizable number of comments to
address, if it took ~8 weeks to address them. That says to me that the
AD was not kept in the loop on the design, as it developed. Did you work
to ensure the AD understaood the directions the document was taking? Did
you ask the AD for advice, or for an advisor to help ensure the document
would be acceptable when it was submitted for their review?

4) If the AD comments were not sizable, then apparently the WG was being
slow to address them. That's nobody's fault but the WG and its chairs.
The WG chair should push the editors to get the work done on a timely
basis, and get commitments for when the work will be done, or close down
the WG.

5) Apparently the IESG had sizable comments to address, if it took ~12
weeks to address them. See bullet points #1 through #4. 

I've had documents take a long time as well, and a lot of the blame
needs to be laid at my own feet.

Keep in mind that if you did not do a good job of preparing the
documents for the review/publish process, you not only caused delays for
your documents, but for everybody else's documents in the review queue.

So ask yourself the question, how well did YOU prepare the document for
the review/publish process?

Dbh
Co-chair, SNMPv3 WG

> Sent: Wednesday, June 11, 2003 11:59 AM
> To: [email-masked]

---

From: Eastlake III Donald-LDE008 <[email-masked]>
Date: Thu, 12 Jun 2003 08:33:21 -0400

There are working group documents of distinctly lesser importance and
individual submisisons of greater importance. Sure, the average time
in queue for independent submissions should be longer than for WG
docs, but it's probably just as important whether a document is
standards track or informational. Also, I think you will find at least
a small effect where larger RFCs take longer.

Donald

PS: Of course, some of the shortest times in queue are for those great
1 April RFCs...

-----Original Message-----
From: Margaret Wasserman [mailto:[email-masked]]
Sent: Wednesday, June 11, 2003 5:45 PM
To: Bernard Aboba
Cc: [email-masked]; [email-masked]
Subject: Re: How long does it take on an average for WG I-Ds?

At 01:37 PM 6/11/2003 -0700, Bernard Aboba wrote:
>AAA WG has had documents in the RFC Editor queue since 12/02 with no
>gating issues.  So it appears that RFC ed queue may be closer to 24 weeks
>or even longer.

Yet, the RFC editor has time to publish individual submissions?

Are there any particular rules regarding prioritization within
the RFC editor?  And does the IETF have any influence over
those priorities?

Margaret

---

From: Bill Fenner <[email-masked]>
Date: Fri, 13 Jun 2003 22:25:29 -0700

>Unfortunately, the RFC editor's *current* publication queue does not tell 
>which states the document has been in before the current "RFC-EDITOR" 
>state.
>
>What I'd really like to know is the effective time rfc-editor takes; 
>excluding time taken in states like REF, AUTH, AUTH48, or IESG.

You mean like

draft-ietf-aaa-transport 20021213 current status EDIT
draft-ietf-aaa-transport 20021213 entered queue at 2002/12/12 (1 days in unknown state)
draft-ietf-aaa-transport 20021213 is version 09
draft-ietf-aaa-transport 20030107 changed from EDIT to AUTH
draft-ietf-aaa-transport 20030107 changed version from 09 to 11
draft-ietf-aaa-transport 20030107 was in EDIT since 20021213 (25 days)
draft-ietf-aaa-transport 20030108 changed from AUTH to EDIT
draft-ietf-aaa-transport 20030108 changed version from 11 to 09
draft-ietf-aaa-transport 20030108 was in AUTH since 20030107 (1 days)
draft-ietf-aaa-transport 20030304 changed from EDIT to IESG
draft-ietf-aaa-transport 20030304 was in EDIT since 20030108 (55 days)
draft-ietf-aaa-transport 20030414 changed from IESG to EDIT
draft-ietf-aaa-transport 20030414 changed version from 09 to 12
draft-ietf-aaa-transport 20030414 was in IESG since 20030304 (41 days)
draft-ietf-aaa-transport 20030424 changed from EDIT to RFC-EDITOR
draft-ietf-aaa-transport 20030424 was in EDIT since 20030414 (10 days)
draft-ietf-aaa-transport 20030612 changed from RFC-EDITOR to AUTH48
draft-ietf-aaa-transport 20030612 was in RFC-EDITOR since 20030424 (49 days)
draft-ietf-aaa-transport 20030613 current status AUTH48 since 20030612 (1 days)

so the time this document spent in EDIT + RFC-EDITOR seems to be 139 days.
See http://rtg.ietf.org/~fenner/iesg/analyze-rfcq.txt .  (This analysis
assumes that the rfc-editor queue reflects the current state of a document
at any given time; I take a daily snapshot of the rfc-editor queue and
log the deltas.)

  Bill

---

From: Joyce Reynolds <[email-masked]>
Date: Fri, 13 Jun 2003 12:23:55 -0700 (PDT)

Amendment to Bob message w.r.t. <draft-ietf-aaa-transport-09.txt>

June 12, 2003: RFC Editor sent "authors 48 hours" notification

2002/12/12   draft-ietf-aaa-transport-12.txt
AUTH48       received new version 4/14/03
B. Aboba, J. Wood
Authentication, Authorization and Accounting (AAA) Transport Profile
Bytes: 87752

Joyce
(for RFC Editor)

---

From: Bob Braden <[email-masked]>
Date: Fri, 13 Jun 2003 13:25:19 -0700 (PDT)

*> From [email-masked]  Fri Jun 13 12:55:56 2003
  *> Date: Fri, 13 Jun 2003 12:31:43 -0700 (PDT)
  *> From: Bernard Aboba <[email-masked]>
  *> To: Bob Braden <[email-masked]>
  *> cc: [email-masked], [email-masked], [email-masked]
  *> Subject: Re: How long does it take on an average for WG I-Ds?
  *> MIME-Version: 1.0
  *> X-AntiVirus: scanned by AMaViS 0.2.1
  *>
  *> > On Wed, 11 Jun 2003, Bernard Aboba wrote:
  *> > > AAA WG has had documents in the RFC Editor queue since 12/02 with no
  *> > > gating issues.  So it appears that RFC ed queue may be closer to 24 weeks
  *> > > or even longer.
  *> >
  *> > 	The true history of: draft-ietf-aaa-transport-09.txt
  *>
  *> The document I was talking about was draft-ietf-aaa-diameter-17.txt
  *>

Excuse me, Bernard, but you said "since 12/02."  The only document in the
queue since 12/02 is the transport document.

Bob Braden