Subject: RE: Input solicited: IETF problem pinpointing at the plenary?

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Wed, 30 Oct 2002 12:11:02 +0100

Inline

> -----Original Message-----
> From: Randy Bush [mailto:[email-masked]]
> Sent: woensdag 30 oktober 2002 7:48
> To: Dean Willis
> Cc: Harald Tveit Alvestrand; [email-masked]
> Subject: Re: Input solicited: IETF problem pinpointing at the plenary?
>
>
> > like 3GPP that ask questions like "This draft was approved by the IESG
> > in June, why don't we have an RFC yet?" and "What do you mean there are
> > security concerns? Why didn't they come up during IETF Last call? Just
> > what are these concerns anyhow, and who raised them?"
>
> the question that i would ask was why did the security problem come up
> and get solved during the wg internal review or ietf last call?
>
I suspect that Randy means (he'll correct me if I am wrong):

  the question that i would ask was why did the security problem NOT come up
                                                               ======
  and get solved during the wg internal review or ietf last call?

Or am I missing something here?
I think the idea is that these things should be seen/uncovered/discussed
in the WG or during WG or IETF Last Call, and not at the IESG when they
have it on their desk. How can we make WGs aware that they must look
at these things right from the beginning?

Bert

---

From: avri doria <[email-masked]>
Date: Thu, 31 Oct 2002 15:42:04 +0100

Bernard Aboba wrote:
>>While part of this
>>is the WG responsibility, part is also the IETF at-large's
>>responsibility.   Why else do we have a IETF Last Call?  I find it
>>baffling, e.g., that the Security folks would allow something to go
>>through IETF Last call without a thorough analysis.
>
>
> In the case of major security problems, in quite a few cases the issues
> were present in very early versions of the draft, and carried all the way
> through the WG to IETF last without being detected.
>
> This speaks to having a review of the security approach very early on.

Oh, it is best if things ae reviewed early and often.  And i thought
that was one of the reasons why the security folks tried so hard to be
be missionaries and get to other WGs early.

What i am saying is that, at the latest, it should be caught in IETF LC
and not wait for IESG review.

a.

---

From: "Jari Arkko (LMF)" <[email-masked]>
Date: Wed, 30 Oct 2002 18:54:58 +0100

Good questions Frank!

> problem. One issue mentioned is the time it takes from
> when a WG says it's done with a draft to when the IESG
> makes its pronouncement.

This is a problem, but it is not the only problem. The
time it takes for WGs to come to a result is also pretty
long in some cases, imho. In fact, looking at the time
scales this feels like the downright most serious
problem we have. Why is this? Probably for somewhat
different reasons than why it takes long for
IESG to make decisions.

As far as the WGs are concerned, some are slow because
of lack of serious good-part-of-your-day contributors.
Some others have creeping featurism -- often these
WGs engage in endless battles over adding or not
adding new features when they should be shipping
base technology. Some other WGs are viewed more as
academic discussion arenas than a timely tool for
producing specifications. And some WGs have underlying
conflicts in terms of what the participants think
are high priority items.What can we do? Some practical
ideas (and some less practical too):

- Make the goals, and the difference of current status
  to the goal more visible. Employ "open issue lists"
  and other tools.
- Consider the meeting schedules, will people do
  more work if they have a twice as many meetings
  (maybe some interim) as they now have? Surprisingly,
  most drafts work happens right before the deadlines.
- Create design teams in order to build an expectation
  of having to do daily or at least weekly work.
- Create pressure for time tables. If you don't deliver
  the RFC -- without errors that IESG can find -- in
  time, you will miss this years RFC publication "train".
  I'm exaggerating a bit, but you get the point.
- Introduce stricter limits for specification size.
  This would silence many of the we-need-my-toy-thing-
  in-the-base-spec people, and fight featurism.
- The IETF has grown a lot, and in many ways the
  consensus approach hasn't grown with it. WGs can
  be big, but still there are only a couple of
  people debating about an issue, and a single
  person's opinion counts much. Could we find
  automated ways to ask for consensus or the humming
  without simultaneously causing an explosion in
  the amount of e-mail?
- Time. In some other standards bodies folks can
  spend a significant fraction of their time in
  working (emphasis on that word) meetings. And
  we get by what, 6 hours a year. We are efficient
  and we know what e-mail means but...
- Parallel work. Allow different people to do
  different things. At the moment we have
  some cases where new work by folks A is
  prevented from being done because the "WG
  is busy" with some other work done by
  folks B.

What about the IESG then? I think it would be wrong
to blame the IESG itself for being slow, or the WGs
for using it as a review tool. I have always gotten
to-the-point feedback from the IESG, and I feel like
they have given me a lot of their time. So they are
doing a good job. But the output from the IESG
is basically determined by the number of ADs, quality of
the input, and the level of involvement. Are the areas
too large, should there be more ADs? How can we increase
the quality of the WG documents? Better work in the
groups of course. But other than that, additional transparency
e.g. in the form of public, written down policies on
extensibility, for instance, would be good. Level of
involvement? Could some of the tasks of the IESG be
delegated? Could WG chairs take a bigger role in
reviewing some of the nits and editorial issues that
sometimes the IESG has to do? ADs use certain
trusted but unofficial experts when looking at specific
things. Should such roles be made more official and
have less of the discussion go through the ADs?

--Jari

---

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Mon, 4 Nov 2002 21:13:48 +0100

> > From: "Frank Kastenholz" <[email-masked]>
> > To: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> > Sent: Monday, November 04, 2002 8:53 AM
> ...
> > Except that company X has invested umpteen million dollars in
> > one proposal and company Y has invested umpteen million dollars
> > in the other one. Your observation means that one of the two
> > companies throws its investment away and reinvests (losing
> > money and time in the market).
> ...
>
> If the additional value to having a *single* standard is that easily
> offset by the development and time-to-market costs, then I think
> it's fair to question (in this hypothetical case) whether it is really
> needed.
>
> A more fundamental question is whether the trend towards
> "late standardization" (after implementation and perhaps even
> deployment) would exacerbate matters by increasing the stakes.
>
Very good question. And it seems to advocate that maybe we should
already at the (early) I-D stage try to pick one solution.

Bert
> Randy Presuhn

---

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Tue, 5 Nov 2002 21:13:41 +0100

I like this thinking very much.

Bert

> From: Edward Lewis [mailto:[email-masked]]
> Sent: maandag 4 november 2002 19:41
> To: [email-masked]
> Cc: [email-masked]
> At 8:08 -0800 11/4/02, Marshall Rose wrote:
> >melinda - no, he's politely commenting on my note to frank.
>
> ;)
>
> I guess I operate by school #1 - that if consensus isn't reached it's
> best to walk away from declaring any solution.  It's happened in
> provreg (we've had about 9 documents[0] so far enter the WG - 1 has
> gone to RFC, 5 are in front of the IESG, 3 have been dropped and just
> 1 still under consideration).  I.e. 33% of our documents to date have
> been killed because no consensus could be reached.  (Whether because
> of "I don't like" or "I don't care" sentiments.)
>
> My rationale is that consensus means that the group has found common
> ground.  If there is no common ground, then we don't have
> interoperability - and that is the single most important feature of
> an IETF standard.  If interoperability isn't the overriding goal,
> then the WG ought to be conforming to some other standards body
> anyway.
>
> As far as "closing the WG" - that's not always the solution.  I've
> seen individual issues (DNS is loaded with them) that fail to achieve
> consensus and then wither away without closing the group.  (I suppose
> that the intent in school 1 was not to always result in a fatal error
> for the WG, but when I first read the message, I didn't recognize #1
> as my solution because of the "closing" phrase.)
>
> [0] 2 are knocking at the door but haven't been allowed in.
> --
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Edward Lewis                                          +1-703-227-9854
> ARIN Research Engineer
>

---

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Thu, 31 Oct 2002 16:06:12 +0100

Here is input from a SNMP/MIB contributor. Not a WG chair though.
I bet there are WG chairs that will recognize this and probably
some will have additional input.

Thanks,
Bert
------------------
I will just write down some observations. Here is what comes to my
mind at the moment. Perhaps this serves as input. (I am not touching
any generic IETF process issues which I think others can do much
better.)

1. MIB modules must be available early in the life cycle of a new
   technology in order to get implemented. Trying to standardize MIB
   modules after vendors have developed and deployed proprietary
   versions is a waste of time.

2. MIB modules are sometimes not of prime interest in WGs and as such
   there is little support in terms of meeting time, chairs pushing
   things forward, deep technical review by the subject experts, ...

3. MIB module production and review are often sequential, which can
   considerably slow down the overall process. Especially the desire
   to integrate new work with existing core infrastructure MIB modules
   (such as IF-MIB or ENTITY-MIB) can cause serious surprises if WGs
   are not in close contact with MIB experts early in the process. In
   fact, it seems to be desirable that MIB modules are developed in a
   joint effort rather than a sequential process.

4. Even though there is considerable review, many MIB modules still
   lack a decent description of the information model behind the MIB
   module and typical use cases. This is especially interesting since
   the lack of a decent description usually also slows down the review
   process.

5. The SMI defines clear rules on how to extend MIB modules which
   protect people from creating interoperability problems with
   previous versions. However, in some cases, a reasonable argument
   can be made that not strictly following some of the rules is
   actually the more reasonable thing to do based on cost/benefit
   analysis. There needs to be a mechanism to make such a decision and
   to apply rules wisely. The fact that it is really tough to get
   changes and clarifications into a MIB module causes many developers
   to cheat in their implementations rather to contribute valuable
   feedback to the WGs.

6. The IETF standards process defines another set of rules. The
   current interpretation that every addition to a MIB module resets
   the IETF standards status to Proposed causes WGs to do things such
   as the creating of the IF-INVERTED-STACK-MIB which technically
   really belongs into the IF-MIB. There needs to be a way to publish
   additions and corrections to MIB modules without resetting their
   overall standards status to Proposed. Another example in this area
   was the addition of some enumerated values to the RMON-MIB to
   support high-capacity RMON.

7. A number of MIB modules have been over designed. This might to some
   extend be an effect of the procedure discussed above which makes it
   hard to add things over time. Sometimes this is just the result of
   trying to accomodate each and every idea. Perhaps there is a need
   to develop metrics that can help to measure the complexity of MIB
   modules in order to help MIB authors to make better trade-off
   decisions.

8. There is a lack of experienced MIB/SMI experts. Since this
   situation can not be easily be changed by the IETF/IESG, it might
   be necessary to make decisions where these limited resources are
   spend most effectively, even though this might mean that some
   protocols will not have standard MIB modules developed early on in
   the process, which has negative implications as outlined above.

---

From: [email-masked]
Date: Tue, 12 Nov 2002 21:08:35 -0500

> some will have additional input.

Actually, I have some good news to report about work in which
this pattern is *not* recognizable.

>    things forward, deep technical review by the subject experts, ...

The IP Storage WG (ips) is close to finishing its current program of
work.  We've produced 4 major protocols (iSCSI, FCIP, iFCP, iSNS) and
a total of 7 MIBs, in parallel.  The MIBs and their protocols tended to
get done around the same time.  In many cases we had to hold up WG
Last Call of the MIB just in case the corresponding protocol WG Last
Call generated changes that affected the MIB.  I hear talk of putting
one or more of our protocols into IETF Last Call at the same time as
its MIB(s) to set an example.

Some things that helped:

- IPS used a technical coordinator structure beneath the WG co-chairs.
	Each technical coordinator was asked to take responsibility
	for ensuring that everything else needed to make the protocol
	useful would be complete when the protocol was, starting with
	the MIBs ... they did as asked, and deserve a lot of credit.
- A good MIB doctor with time and patience was an immense help.  I
	don't know how we'd have completed as many MIBs in the time we
	did without Keith McCloghrie's support.
- WG chair interest and support makes a big difference.
- Between the above, it became part of the ips wg environment that
	development of the protocol and its MIB at the same time was
	expected - nobody seriously considered protocol development
	w/o MIB development in the WG.

So, it can be done, but it took a lot of work from a lot of people
(those noted above, as well as the MIB authors who undertook the work).

FYI/Thanks,
--David (ips wg co-chair, Elizabeth Rodriguez is the other co-chair)

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953 **NEW**     FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: "Wijnen, Bert (Bert)" <[email-masked]>
Date: Wed, 13 Nov 2002 10:59:04 +0100

Thanks for this report David.
That is good news.
Of course you were very lucky with one of the best MIB
Advisor we can think of in the IETF. ANd I know he was
motivated to get good MIB modules in this technology area.

I also thank you for pointing out that indeed the WG chairs
play an important role in this in that they must motivate the
WG that the "management part" of the work is as important
as the protocol itself.

I would also like to take this opportunity to encoureg WG chairs
to contact me (or the OPS ADs in general) and/or your own AD
to ask for MIB advisor help if such is needed. It is a scarce
resource, but we will try to find help.

Thanks,
Bert

> From: [email-masked] [mailto:[email-masked]]
> Sent: woensdag 13 november 2002 3:09
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: RE: Input solicited: IETF problem pinpointing at the plenary?
>
>