Subject: Proposal for Structuring New WGs

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Tue, 22 Apr 2014 17:30:20 +0000

Hi,

I¹ve been thinking about how some Wgs get stuck and others loose steam and
would like to propose an optional way to structure new WGs.

I think the delay or failure of many WGs can be attribute as the way we
IETF structures WGs.

When a WG is formed there is lots of energy, many drafts, people with
running code (yes, these people still exist) and great participation. But
then, the IETF step on the brakes by following a cookie cutter approach of
asking for a problem statement draft, some form of architecture draft and
produces, most often than not, a set in stone charter.  This alone makes a
boatload of people leave the WG effort. Some of these folks have even
running code (I¹ve been there) or drafts that many folks are interested
but can not be discussed until the current charter is done.

And the WG struggles on and on because of the very fact that some of the
folks with energy that could drive the WG will not wait on the sidelines
for 2 years until rechartering so that maybe, just maybe, their draft can
get on the charter.

My proposal is that certain WGs should be formed with some overall charter
and the definite charter should be written _after_ say, two official WG
meetings, where lots of discussion on mailing list and presentations
happen. Then the charter becomes the most important drafts people are
interested. In this case for example, the problem statement becomes in
fact those drafts. 

Maybe this is already allowed but it is not known or not encouraged. At
least if possible it would be good to experiment with that approach in
some of the popular or struggling WGs.

Thanks,

-Reinaldo

---

From: Mary Barnes <[email-masked]>
Date: Tue, 22 Apr 2014 13:04:08 -0500

> but can not be discussed until the current charter is done.
[MB] AFAIK there is no requirement to follow this cookie cutter approach.
It makes sense for some WGs and not for others. The STOX WG in RAI is a
good example where some more mature work (based on running code) was
brought in and then refined within the WG. [/MB]

> for 2 years until rechartering so that maybe, just maybe, their draft can
[MB] Yes, there is very frequently an energy issue. The problem I see is
that we don't do a very good job as chairs of recognizing when that has
occurred and managing it.  IMHO, it's up to the chairs to ensure there's a
plan to drive the work. This gets us back to the perennial topic of
deadlines - it's normal behavior for many of us to not get something done
until right before a deadline.  There's absolutely nothing that precludes
chairs from setting  earlier deadlines and intermediate deadlines between
f2f meetings. And, we have found weekly one hour calls to be very helpful
in the CLUE WG to resolve specific issues and focus on key drafts. If
people can't meet deadlines, then the chairs have option of recruiting new
document editors or worst case working with the ADs and shutting down WGs
We've done that in the RAI. So, in my opinion, we don't need a new WG
structure or process. We just need to be more diligent in managing WGs (and
perhaps doing a better job of choosing chairs that have the time to
dedicate to the work other than a few weeks before and after f2f meetings).
[/MB]

> interested. In this case for example, the problem statement becomes in
[MB] Isn't the above the current model we already have with Bofs? [/MB]

---

From: Behcet Sarikaya <[email-masked]>
Date: Tue, 22 Apr 2014 13:07:57 -0500

Hi Reinaldo,

In Multimob, we followed the approach of directly going into solutions as
you seem to advocate.

However later on we realized that some requirements work needed to judge on
the solution proposals because for almost all charter items we had more
than one solution proposals.

In the end we ended up not doing any requirement work but we still got
delayed due to the solution candidate selection process that took longer
than expected.

Regards,

Behcet

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Tue, 22 Apr 2014 18:20:57 +0000

I think this is a bit different. IMO during BOF people are really trying to sell the idea but people really converge after the WG is officially formed and go-ahead is given.

From: Mary Barnes <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 22, 2014 at 11:04 AM
To: Cisco Employee <[email-masked]<mailto:[email-masked]>>
Cc: WG Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Proposal for Structuring New WGs

---

From: "Black, David" <[email-masked]>
Date: Tue, 22 Apr 2014 14:40:58 -0400

That depends on whether a BoF is intended to be WG-forming.  A WG-forming BoF is supposed to come up with a charter.  There could be some preceding exploratory non-WG-forming BoFs, although with some care, as it's a short distance from there to a world in which an effort can take more than 2 BoFs to come up with the charter (e.g., experiences in the recent past with "bar BoFs" having quite a bit of formality and organization).

I've definitely seen the scenario that Reinaldo describes - a WG formed w/a head of steam to go do "something" in an area, but it's not exactly clear what that "something" should be ... so a lot of time and effort are expended on things like a problem statement, framework, architecture, etc. towards figuring out what that "something" ought to be ... and a lot of people lose interest along the way :-(.

Thanks,
--David

From: WGChairs [mailto:[email-masked]] On Behalf Of Reinaldo Penno (repenno)
Sent: Tuesday, April 22, 2014 2:21 PM
To: Mary Barnes
Cc: WG Chairs
Subject: Re: Proposal for Structuring New WGs

---

From: Michael Richardson <[email-masked]>
Date: Wed, 23 Apr 2014 10:52:12 -0400

> but can not be discussed until the current charter is done.

I don't think that the IETF forces this in a structural way.

I think that this occurs as an import from other SDOs, from the discredited
software 'waterfall' method, etc.

I think that people are naturally much more agile when left to run on their
enthusiasm, and I think that our mantra of running code and rough consensus
was attempting to channel that.  However, we do want to have some milestones
that make sense, and perhaps the right thing here would be to come up with
a newer, more agile, template charter/milestones that captures the desired
iterative nature that we really want.
(HEY, IAB people, I mean, *YOU*... you guys got a retreat about to happen?)

--
Michael Richardson <[email-masked]>, Sandelman Software Works
IETF ROLL WG co-chair.    http://datatracker.ietf.org/wg/roll/charter/

---

From: Marc Blanchet <[email-masked]>
Date: Wed, 23 Apr 2014 11:06:47 -0400

> iterative nature that we really want.

agree on the agility/iterative idea. but it is a fine balance between too structured and not enough structured...

Marc.

---

From: Mary Barnes <[email-masked]>
Date: Wed, 23 Apr 2014 10:06:54 -0500

> (HEY, IAB people, I mean, *YOU*... you guys got a retreat about to happen?)
[MB] There's absolutely nothing in the current model to prevent WG chairs
from doing more detailed project planning.  I don't think we need a process
in place as the task can be straightforward for some WGs. And, chairs are
totally free to implement a process that they feel does the job.  I
maintain a spreadsheet for my WGs that includes intermediate milestones,
since we almost always iterate documents through the WG milestones - i.e.,
multiple WG last calls including what I call a pre-WGLC.  This isn't a
tool or process problem, this is an issue with WG chairs not managing the
process we have. [/MB]

---

From: Michael Richardson <[email-masked]>
Date: Wed, 23 Apr 2014 16:54:12 -0400

Mary Barnes <[email-masked]> wrote:
    > [MB] There's absolutely nothing in the current model to prevent WG chairs from
    > doing more detailed project planning.  I don't think we need a process in place
    > as the task can be straightforward for some WGs. And, chairs are
    > totally free

I'm not suggesting any process changes, but rather a reference
design/template/running code of a more agile/iterative charter + milestones.
Something that has been pre-reviewed by I* types.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Wed, 23 Apr 2014 22:07:23 +0000

Thanks Michael for the good summary, I fully agree. As I mention the
process today might allow these agile WGs, but we need the backing, tools
and encouragement to actually make this a reality.

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Wed, 23 Apr 2014 22:16:51 +0000

if you ask some developers to reach an agreement on the definition of Firewall, it will never happen. But if you ask them to code a firewall product, they can deliver one to you in no time. And there, IMHO,  is the main problem of the way most WGs are structured.

From: <Black>, David <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, April 22, 2014 at 11:40 AM
To: Cisco Employee <[email-masked]<mailto:[email-masked]>>
Cc: WG Chairs <[email-masked]<mailto:[email-masked]>>
Subject: RE: Proposal for Structuring New WGs

I’ve definitely seen the scenario that Reinaldo describes - a WG formed w/a head of steam to go do “something” in an area, but it’s not exactly clear what that “something” should be ... so a lot of time and effort are expended on things like a problem statement, framework, architecture, etc. towards figuring out what that “something” ought to be ... and a lot of people lose interest along the way :-(.

Thanks,
--David

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 23 Apr 2014 22:09:36 -0400

On 4/23/14 6:16 PM, Reinaldo Penno (repenno) wrote:
> if you ask some developers to reach an agreement on the definition of
> Firewall, it will never happen. But if you ask them to code a firewall
> product, they can deliver one to you in no time. And there, IMHO,  is
> the main problem of the way most WGs are structured.

And when two of them do that, they probably will not agree that what the 
other has developed *is* a firewall.

It is problems like that which have led to a request to start with 
requirements.

There is probably a happy medium somewhere, but it isn't always in the 
same place.

	Thanks,
	Paul

> From: <Black>, David <[email-masked] <mailto:[email-masked]>>
> To: Cisco Employee <[email-masked] <mailto:[email-masked]>>
> Cc: WG Chairs <[email-masked] <mailto:[email-masked]>>
> I’ve definitely seen the scenario that Reinaldo describes - a WG formed
> w/a head of steam to go do “something” in an area, but it’s not exactly
> clear what that “something” should be ... so a lot of time and effort
> are expended on things like a problem statement, framework,
> architecture, etc. towards figuring out what that “something” ought to
> be ... and a lot of people lose interest along the way :-(.
>
> Thanks,
> --David
>

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 24 Apr 2014 14:33:34 -0400

On Wed, Apr 23, 2014 at 04:54:12PM -0400, Michael Richardson wrote:
>     > doing more detailed project planning. ??I don't think we need a process in place

I'd suggest there are two different issues being discussed here:
- Project management of WG tasks.  The above seems to fall in this category.
- Process that, when injected into WG work, is toxic to WG throughput.
  (Whether the process was needed and why is I think what kicked off the
   thread.)

FWIW, I'm a fan of "best practices for running a WG".  The one thing I'll
note is that the lag-time of various individuals in being able to
participate in the group's work can complicate certain forms of "agile". :-)

(I keep telling myself in my copious spare time that I'd analyze WG mail
list contribution activity surrounding the conferences and certain holidays,
but keep failing to find such copious spare time.)

The process issue is a bit different, IMO.  When architecture, problem
statements or use cases are thrown at an issue it's typically because
there's either enough difference in view as to what should be worked on or
its output. (The recent "here's three firewalls and no one does the same as
the others example elsewhere in-thread.)  Such requirements documents are
typical mitigation mechanisms elsewhere in the hardware/software industry so
asking for them isn't bad in itself.

What's problematic is when those documents either "circle the drain" because
no one can agree with each other or they don't get effort because people
aren't interested in working on them and just want to start writing code.

Fixing this is hard.

-- Jeff

---

From: Loa Andersson <[email-masked]>
Date: Sat, 26 Apr 2014 10:26:04 +0200

Folks,

What is the driver behind this i there a wildly felt need to for
giving each and every working the same structure?

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert                          [email-masked]
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: Benoit Claise <[email-masked]>
Date: Mon, 28 Apr 2014 10:16:05 +0200

Hi Reinaldo,
> but can not be discussed until the current charter is done.
Yes, this is a real problem.
While discussing this issue, Spencer Dawkins reminded me of RFC 5434:

    In many cases,
    however, the intent is to form a WG.  In those cases, the goal of the
    BOF is to demonstrate that the community has agreement that:

       - there is a problem that needs solving, and the IETF is the right
         group to attempt solving it.

       - there is a critical mass of participants willing to work on the
         problem (e.g., write drafts, review drafts, etc.).

        - the scope of the problem is well defined and understood, that
         is, people generally understand what the WG will work on (and
         what it won't) and what its actual deliverables will be.

       - there is agreement that the specific deliverables (i.e.,
         proposed documents) are the right set.

       - it is believed that the WG has a reasonable probability of
         having success (i.e., in completing the deliverables in its
         charter in a timely fashion).

Special attention to the third bullet point above.
If the problem is well understood enough for a working group to be 
chartered, how much more work does a working group need to do, in 
defining the problem?
Maybe we should apply this rule in a stricter manner. You wrote "When a 
WG is formed there is lots of energy ", I would observe that when a WG 
is about to be created (BoF), there is even more energy ... to get the 
WG created. We should really capitalize on it.

Regards, Benoit

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 28 Apr 2014 10:32:25 +0100

Hi Benoit,

Thanks for flagging this piece of text...

>>    what it won't) and what its actual deliverables will be.

You said...

> chartered, how much more work does a working group need to do,
> in defining the problem?

I have seen a number of cases recently where the problem was *not* well enough
understood to charter a working group. True, if you stopped any one person in
the corridor, they might have said that *they* understood the problem well
enough, but it was clear to me in these cases that there was not even a rough
consensus of agreement of understanding of the problem.

In these cases there are three option:
1. Say no. Ask for more work agreeing the problem and hold
   another BoF later.
2. Form a WG to work to agree on the problem and recharter
   later.
3. Take a punt. Form the WG to do real, solutions work. 

I have seen the third of these go horribly wrong: many people developing
different solutions in the same space to do subtly different and conflicting
things.

The first option, on the other hand, seems to be more blocking of progress than
the second option, and so I tend to adopt the second option. That option brings
the work into the IETF, but requires that effort is expended developing the
problem space and scoping the solutions space.

So, when Reinaldo bemoans that:

> the IETF step on the brakes by following a cookie cutter approach of
> produces, most often than not, a set in stone charter.

I find that this is often a necessary step that has been "borrowed" from the BoF
process. That is, it is legacy work that should have been done before the WG was
formed, but which we have decided to let slip until after the WG has been
formed. We have chosen to form the WG because it is clear that we have a topic
that has significant interest and is in scope for the IETF, but we also
recognise that the problem is not well-defined or understood.

But where I am repeatedly frustrated is the length of time the WG spends in this
state. If, as is widely claimed, the problem *is* well understood to the extent
that running code exists, I do not understand why it takes multiple IETF cycles,
and often multiple years, to document the problem.  If so many people are
interested and working in the space, why is none of them capable of writing a 20
page document that clearly and concisely describes the problem? If there is a
space where many people are writing code, why is it so hard to get consensus on
the problem statement?

No, I know two answers to my questions both predicated on the assumption that
writing a short document is not problem statement:

a. It is not as exciting as writing code.
b. It is not beneficial to all parties to document the problem space.

The second of these may be me being cynical and jaundiced. But it seems to me
that if you have written code and you find that the problem is defined different
from what you have coded this may be unfortunate for you. At one end of the
spectrum it could cost a lot of money; at the other end of the spectrum it is an
attack on someone's baby.

So my question back to Benoit, and rephrasing his question...

If the problem is well understood enough for a working group to be chartered,
why does it take that working group so long to write a tight and clear document
defining the problem?

Cheers,
Adrian

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Mon, 28 Apr 2014 11:16:39 +0000

________________________________
From: Benoit Claise (bclaise)
Sent: Monday, April 28, 2014 1:16 AM
To: Reinaldo Penno (repenno); 'WG Chairs'
Subject: Re: Proposal for Structuring New WGs

Hi Reinaldo,

Hi,

   In many cases,

Special attention to the third bullet point above.
If the problem is well understood enough for a working group to be chartered, how much more work does a working group need to do, in defining the problem?

[RP] My answer is none. If the problem space is well understood why do we need "problem statement" drafts? The charter should suffice and we can hit the ground running. But that is certainly not the case that's why I proposed agile WGs that work around such blocking issues and get work done.

Maybe we should apply this rule in a stricter manner. You wrote "When a WG is formed there is lots of energy ",

[RP] I replied to this point to another person. In my view it is a different kind of energy and priority as I discuss below.

I would observe that when a WG is about to be created (BoF), there is even more energy ... to get the WG created. We should really capitalize on it.

[RP] Right, this is key: "to get the WG created". During BOF phase what I notice is that getting the WG formed takes priority over anything, all other issues are brushed aside in the hopes to get the WG formed and getting work done. I mean, otherwise how come in a BOF we can get 100 people to agree on a single set of ideas (really??) and later have endless threads on the list.

[RP] I do not want to go in a tangent here but asking a huge audience if they all agree that the scope of problem is well understood and getting no disagreements would be a red flag right there.

Regards, Benoit

Thanks,

-Reinaldo

---

From: Benoit Claise <[email-masked]>
Date: Mon, 28 Apr 2014 14:58:16 +0200

> why does it take that working group so long to write a tight and clear document
Because once the WG is created, it's more fun to design a solution than 
to document the problem statement.

Let's go back to your 3 categories:

All 3 are viable options.
For category 2, I'm now in favor of adding in the charter: if the WG 
can't agree upon a problem statement by date D, it will be closed

Regards, Benoit
> .

---

From: "Black, David" <[email-masked]>
Date: Mon, 28 Apr 2014 09:44:09 -0400

Selective excerpts from Adrian's note ...

> 2. Form a WG to work to agree on the problem and recharter later.

> space where many people are writing code, why is it so hard to get consensus on

I know of at least one case, with which Adrian is quite familiar ;-), where
there's a good 20/20 hindsight case to be made that the new WG was asked to
merge two problems into one - I still have some of the scars from what it took
to write the resulting problem statement (which is now at the RFC Editor).

NB: hindsight is always 20/20; the hard part is making the decision w/o that.
IMHO, the approach of trying to do all of the problem unscrambling/clarification
in one WG seemed like a plausible idea at the time ...
... and there had been some even broader prior BoFs and non-BoFs.

Thanks,
--David (an author of more than one WG problem statement doc)

> -----Original Message-----
> From: WGChairs [mailto:[email-masked]] On Behalf Of Adrian Farrel
> Sent: Monday, April 28, 2014 5:32 AM
> To: 'Benoit Claise'; 'Reinaldo Penno (repenno)'; 'WG Chairs'

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 28 Apr 2014 15:49:05 +0200

My problem with these discussions is that they tend to be very abstract.
Hence, it is difficult to draw a useful conclusion from the discussion
other than everyone has their unique experience with a number of groups.

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 28 Apr 2014 15:25:35 +0100

> other than everyone has their unique experience with a number of groups.

Yeah, life is hard, isn't it?
I don't like using individual people's screw-ups (even my own) as examples in
public debate.

But let's look at ACE.

It has been obvious since before the ACE working group was started at the
beginning of December last year, that there was a group of people very
determined to work in this area. A BoF was held in London. More work was needed
to refine the problem statement and the area of work.

Now (end of April and so five months on) there is a draft charter (quite
lengthy) that still needs review and discussion. The charter includes a lot of
text that scopes the problem, but still includes the first deliverable (of a
total of two) as "Produce use cases and requirements."  It is clear to me,
therefore, that some aspects of the problem statement are captured in the
charter text, but that there is not sufficient consensus and agreement on the
problem statement to move straight to solutions work.

If I was a developer writing security code for applications in constrained
environments, would I feel annoyed that the IETF was still fiddling with this
and not moving ahead faster? Working with my team of developers or my online
community, I could write a lot of code in five months. And it will presumably be
more months before the working group (if formed) get around to adopting
solutions documents.

As it happens, I think the ACE community has done the right thing. I might wish
they had done it faster and more focussed, but that was clearly their choice and
not the influence of the wider IETF community. And that is my broader point...

Stated with more bluntness than is likely to be good for me: When people
complain that the IETF is slow to adopt work or slow to publish work, have those
people actually been contributing constructively to the consensus process? Have
they contributed text? Or were they part of the problem?

Cheers,
Adrian

---

From: Spencer Dawkins <[email-masked]>
Date: Mon, 28 Apr 2014 09:26:35 -0500

> giving each and every working the same structure?

Speaking only for myself, when I feel wild needs, it's pretty much never 
this one :D

More seriously, and I'm conjecturing, I suspect that this is a 
combination of people cloning what worked in previous WG charter efforts 
and people who are active in other SDOs cloning what other SDOs do in 
the IETF.

I don't know what other ADs tell their BOF requestors, but most of the 
BOF requests I see propose a charter that includes the investigation-ish 
sort of documents.

Spencer

---

From: Sandra Murphy <[email-masked]>
Date: Mon, 28 Apr 2014 13:36:51 -0400

> why does it take that working group so long to write a tight and clear document

My answers (related):

Abstraction is hard for humans.  A definition is an abstraction of the important attributes of a problem, and hence hard.

Writing is hard for humans.  I read someone's quote once that writing was the greatest spur to clear thinking.(*)  So the process of trying to write a description of the problem exposes any fuzzy thinking.  (I used to think that one thought clearly, and then one wrote clearly.  But I've experienced the feedback process myself.)

This is hard for one human and mutiplicatively hard for multiple humans in collaboration.

--Sandy

(*)Don't know my original source. Quick web search yields lots of similar comments. On Writing Well by William Zinsser, among many others.

---

From: "Reinaldo Penno (repenno)" <[email-masked]>
Date: Mon, 28 Apr 2014 19:56:41 +0000

Right!

I tried an analogy with Firewall development but maybe this is a better
one:

You can get dozens of nations to search for a missing plane, but if you
have asked them before starting the search to write a ³problem statement²
document they certainly would still be at it trying to define what is the
exact definition of ³missing².  And I think due to our very nature this
problem amplified.

Therefore my proposal for more agile WGs.

Thanks,

On 4/28/14, 10:36 AM, "Sandra Murphy" <[email-masked]> wrote:

>>chartered,
>Abstraction is hard for humans.  A definition is an abstraction of the
>important attributes of a problem, and hence hard.
>
>Writing is hard for humans.  I read someone's quote once that writing was
>the greatest spur to clear thinking.(*)  So the process of trying to
>write a description of the problem exposes any fuzzy thinking.  (I used
>to think that one thought clearly, and then one wrote clearly.  But I've
>experienced the feedback process myself.)
>
>(*)Don't know my original source. Quick web search yields lots of similar
>comments. On Writing Well by William Zinsser, among many others.