Subject: Re: engineering discipline WAS: stats from the text conferencing experiment

From: Randy Bush <[email-masked]>
Date: Sat, 30 Nov 2002 21:11:09 -0800

> In general, I think that area meetings are good to general issues
> that affect the area, but they are too big to serve as a useful
> forum for people to meet each other.

scaling chapter 42 paragraph 11

and i suspect the book does not have a happy ending.  but then,
i am a pessimist.

> Maybe try to get WG chairs from an area together at the IETF?

ok.  i am pragmatic.  how about an ops chairs meeting in sf?  how
about monday dinner?

> Or, make the area chairs mailing lists more active, and raise
> general area issues on that list?

well, we're singin' duet here, but no one else is picking up the
harmony.

> Or, perhaps, when there is a situation that involves two WGs the
> ADs could point the WG chairs at each other, ask them to work out
> a resolution and get back to the AD?

happens very often.

>> most human cultures seem to respond to the strains of scale by
>> adding process.  of course, this makes them more vulnerable to
>> stultification, take-over, and filibuster.
> Yes, I agree that seems to be what normally happens, and I suppose
> it may be unavoidable...

it's easy to make a hide-bound bureaucracy.  the challenge is how
to make scale work in the ietf without losing the ability to
produce quality relevant technology.

randy

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 01 Dec 2002 06:45:29 -0500

>ok.  i am pragmatic.  how about an ops chairs meeting in sf?  how

I'm there! :-)

>produce quality relevant technology.

Agreed.  Are there examples of groups that have successfully
made this transition?  Or do we need to chart new territory?

Margaret

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 1 Dec 2002 07:56:20 -0800 (PST)

> made this transition?  Or do we need to chart new territory?

I believe that there are standards bodies, such as IEEE, that have handled
growth with less deterioration in quality and timeliness. However, they
also have considerably better tools for keeping track of comments,
an intellectual property policy that does a better job of inducing
disclosure, and a process that forces participants to read the documents
(or abstain) or lose their voting rights. And they also tend to be more
conservative about scope expansion.

Note - the IEEE has an executive committee but it only takes on a
fraction of the duties of the IESG.

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 01 Dec 2002 12:30:12 -0500

>growth with less deterioration in quality and timeliness.

I've heard several other people point to the IEEE as a good model
for a very large, well-organized standards body.

Do you know much about how they are organized?  I've never been, but
the things I've noticed from the outside are:

         - Corporate membership
                 - Membership is in particular sub-areas like
                   802, not in the whole organization
         - Fairly expensive
         - Documents cost money or are unavailable to non-members

Do you have any perspective on what the IEEE is like on the inside?

>fraction of the duties of the IESG.

As bodies grow, it is usual for the central "government" to have
fewer and fewer duties...  Which duties have remained with the
executive committee in the IEEE?  Who does the rest of the work
that the IESG currently does in the IETF?

Margaret

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 1 Dec 2002 10:05:36 -0800 (PST)

> Do you know much about how they are organized?

There is the executive committee, then the chairs of the individual groups
(IEEE 802.1, 802.11, 802.15, 802.16, etc.), and task group chairs (IEEE
802.11f, 802.11i, etc.) In IEEE 802.1, the chair (currently Tony
Jeffree) operates more like a WG chair with several work items (IEEE
802.1aa, 802.1w, etc.) In IEEE 802.11 the chair (Stuart Kerry) tends to
function more like an Area Director than a chair, since IEEE 802.11
is broken up into Task Groups, each of which has a chair, and its own
work items.

At the IEEE 802 plenary meetings, all groups are present although they may
not meet at the same hotel, and time can be scheduled for inter-group
meetings. For example, IEEE 802.11 could meet with 802.1aa, 802.11i could
meet with 802.11f (not a particularly productive discussion, that), etc.

The process of working group formation occurs via the Project
Authorization Request (PAR). This is similar to the WG chartering
process, but it seems to operate more expeditiously. Typically there is a
"Call for Interest" meeting (similar to a BOF) and then a Study Group is
formed to scope out the field and write the PAR. The Study Group typically
operates for no more than 6 months.

I'd note that the PAR process seems considerably more efficient than the
WG formation and requirements process in general, though I have seen
examples where IEEE 802.11 task groups have gone astray due to lack of
structure that might have been available had requirements been done first.

> Do you have any perspective on what the IEEE is like on the inside?

I've been a voting member of 802.11, and a participant in 802.1. Here are
some observations on several issues:

- Funding
- Throughput/latency
- Tools
- IPR policy
- Participant accountability

Funding

Just as IETF has no corporate existence and is funded by ISOC, IEEE 802
exists within the IEEE corporate umbrella. Both encourage corporate
memberships to supply funding, and the memberships are fairly
expensive. There are tensions in the relationship between IEEE and
IEEE 802 just as there are issues in the relationship of ISOC and IETF.
However, it's not clear to me that the funding issues affect the
operation of the IEEE 802 standards process any more than ISOC finances
affect the operation of the IETF standards process.

Throughput/latency

This seems to vary widely by sub-group. For example, IEEE 802.1 seems to
regularly put out high quality standards in 18-24 months, while IEEE
802.11 takes much longer (IEEE 802.11 took 7 years, I believe). Having
attended both, I'd observe that the chairs and editors of IEEE 802.1 (Mick
Seaman and Tony Jeffree) have many years of experience, keep tight
reins on scope expansion, and seem to benefit from having a relatively
tight knit group that seems to have been together for quite a while. So
there seem to be definite benefits from having a highly competent "old
guard" in charge.

Tools

Tools usage varies widely within IEEE 802, but 802.11 task groups tend to
use database tools to improve the efficiency of comment resolution. The
results are quite impressive -- groups can resolve and track hundreds
of comments from ballots in a way that would not be possible without the
tools. Tools is an area where the IETF has clearly been under-investing.
While IETF WG are using various tools today, as was discussed at the
Tools meeting for WG chairs at IETF 53, we could do more. There is now a
Bugzilla database set up for use by WG chairs, and perhaps this will lead
to more widespread usage (and understanding of requirements).

IPR policy

The IEEE IPR policy does seem to do a better job at requiring disclosure,
earlier in the process. Disclosure is more widely required -- it's not
just for "contributions", as I understand it, but a general obligation of
attendees.

Participant accountability

In IEEE, the ability to vote is tied to responses to ballots. If you don't
send in comments, (or at least abstain), you lose your right to vote. Now
the voting process has many bad side effects (block voting, wierd
political maneuvers, etc.) but it does result in wide reading of documents
within the voting membership, and a deluge of comments (hundreds or even
thousands). In IEEE you do not generally have many last minute issues
cropping up after passing of final ballot -- anyone who expects to have a
say in the standard will have spoken up way before that time.

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 1 Dec 2002 10:27:48 -0800 (PST)

> that the IESG currently does in the IETF?

To first order, I'd argue that an IETF reorganization should start from
the principle that the IESG is a "last resort" rather than the
owner of the "document quality" problem.

One of the first principles of quality control is that in the long term,
there should be no "quality department" -- everyone is responsible for
quality. Similarly for testing -- a software development organization that
vests all responsibility for quality in testers (as opposed to the
developers) produces poor quality software.

So if we can imagine an IETF in which the IESG does not exist, and yet
it is still possible to produce quality documents in a timely way, then
we can add the IESG back as a final quality check, but with more
reasonable expectations about its responsibilities relative to those of
the WGs.

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 01 Dec 2002 14:45:37 -0500

This sounds right to me.  And, it fits with my theory that the
_real_ problem in the IETF is that we need more accountability
at the lower levels -- WG members, authors, editors and chairs.

I'm just having trouble figuring out how we can get this to
happen.

I think that we need to change our culture -- particularly
regarding how people review and critique each others work.

Sometimes it feels like we're just going through the motions --
turning the crank and churning out documents...  Someone proposes
a document, a few folks read it, the document gets accepted as
a WG work item by default (folks seem to say "yes" to anything
the don't find horribly offensive), then the document sits until
the author or someone else gets impatient about how long its been
sitting around on the WG's plate.  Then, the document gets sent
to WG last call, frequently with few if any comments, and it
gets hurled over the wall in the IESG...

I'd ask "what's wrong with this picture?", but maybe it would
make more sense to ask "what's right with this picture?".

I don't think that I'm the only person who has seen this happen
or (I'm ashamed to say) has participated in this type of process...
  So how can we make it stop?!

Margaret

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 1 Dec 2002 22:08:59 +0100

> reasonable expectations about its responsibilities relative to those of

Ah, what a fun discussion.
This would require ways to disseminate knowledge about "protocol 
quality" within the community (and agree on what's knowledge in the 
first place).
Clearly a challenge for a loose organization like the IETF.

Maybe John's proposal to have the IAB set "policies" is going in the 
right direction here.
Except that protocol design is still much more of a craft than a 
science, and it will be hard to codify clue into "policies".
(See RFC1958 for a modest attempt to do this; of course, there are many 
more documents like RFC2914 on specific issues.)
Still, right direction.  We need to collect and agree on a body of 
these.

The next step then of course is a seminar circus and some expensive 
certification process ("certified IETF clued protocol design 
professional" or some such).  You would need sign-off from at least two 
CICPDPs before you could submit a protocol to the IESG.
Somehow I'm not so sure about this...

One reason the IETF meritocracy works as well as it does is that it is 
much easier to recognize a clued individual than to agree/disagree on 
every specific belief of that individual.
To scale better, we can at least agree on some of these, though.
Time to re-spin RFC1958?  On a somewhat grander scale?

Gruesse, Carsten

---

From: Melinda Shore <[email-masked]>
Date: Sun, 01 Dec 2002 16:26:22 -0500

> every specific belief of that individual.

I'm not sure about that - I think it's pretty clear that
there are individuals with microclue (deep understanding of
a narrow range of technologies but not necessarily the
entire context) and individuals with macroclue (deep
understanding of the way the pieces interact but not
necessarily the nasty, necessary bit-by-bit details) and I
think sometimes a clueful person of one ilk is misunderstood
to be a clueful person of the other.

I'm also less sure that it's a meritocracy; one thing that's
happened with the ascendancy of the professional
standardizer is that there are now people whose full-time
job it is to work on standards, while there are many, many
engineers for whom it's an ancillary activity.  The
professional standardizers tend to come from the telecomm
companies and may not have a particularly extensive
familiarity with IP.  In a volunteer organization this can
make a very big difference.  In part the question comes down
to how to create greater personal investment on the part of
individual participants in good architecture, or how to keep
the flame alive.  Maybe more needs to be done with
education, but I'm really not sure.

Melinda

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 1 Dec 2002 23:16:52 +0100

> Maybe more needs to be done with

I'm sorry if the educator in me shines through, but:
In quality 101 one learns that there is no quality plan without an 
education plan.
Creating motivation for education certainly is as much an element of 
that plan as is obtaining quality education.
In a random company, I can foster education by promoting people that 
take the extra step.
(Hmm, maybe the CICPDP is not such a stupid idea after all?  It seems 
to work for CISSPs...  No, I'm not serious.)

Oh and I agree there is a need for clue diversity.  Both wrt macroclue 
and microclue, and wrt different fields of clue contexts.
So maybe we never can make redundant a body with diverse clue like the 
IESG.
(Some of that clue, of course, comes directly from being subjected to 
the unabated storm of incoming documents...)

You do raise the right question: what would motivate the average IETF 
contributor to increase their clue?
And how can we (IETF as a community) help?

Gruesse, Carsten

---

From: "James Kempf" <[email-masked]>
Date: Mon, 2 Dec 2002 10:44:59 -0800

> at the lower levels -- WG members, authors, editors and chairs.

Most WG members want WG chairs to be "above the fray." In particular, this means
that they do not want WG chairs to have technical opinions about matters before
the WG. Basically, the current popular model of WG chairs is that they are there
to facilitate the process and nothing more.

So while I can see WG chairs being more proactive about issues such as writing
quality, I have a difficult time seeing how it would be possible to have WG
chairs involved in enforcing the kind of technical quality the IESG expects
without a major change in the expectations of WG members about the chair's role.
In some cases, the WG chair's opinion might simply be an alternate to what other
WG members want, and, without explicitly saying that "this is want the IESG
wants" it is difficult to get the WG to agree. Saying that makes it clear that
either they agree or the WG documents don't got to RFC.

BTW, I think this situation may have understandably evolved because, in the past
I'm told, there were technically strong WG chairs that essentially did not allow
any technical opinion except theirs to be discussed.

            jak

---

From: Randy Bush <[email-masked]>
Date: Wed, 27 Nov 2002 15:33:22 -0800

> The type of discipline I was talking about, though, is not
> mailing list discipline, but engineering discipline.
>
> How can we convince a large group of people to:
>
>          - Minimize design complexity to keep implementation
>                  costs low and reliability high?
>          - Remove features that are not widely implemented
>                  or deployed, instead of arguing that they
>                  should be left in because they _might_ be
>                  useful?
>          - Understand the problem statement, and design the
>                  simplest solution that solves the problem?
>          - Do the work that needs to be done to finish the
>                  base protocol, instead of inventing optional
>                  features or running off and examining every
>                  "interesting problem" that comes along.
>                  [This most often manifests itself in a lack
>                  of interest in writing or reviewing MIBs.]
>
> We've tried, in the past, to force some discipline through
> charters.  However, the way the IETF is organized, there
> is no way for WG chairs to hold the line.  People who want
> to do unchartered work can:  ask for a BOF, submit the
> work as individual drafts to the RFC editor, bring it
> to the IESG through a different AD, etc.
>
> What would really work is if we could foster an environment
> where engineering discipline is one of our core values, and
> where it is "cool" to make decisions that simplify and
> focus our work, rather than our current status where it is
> cool to invent a whole new protocol and/or identify an
> "interesting" problem and pursue it.
>
> Are there any ways that we could do that?

that more folk are starting to say stuff like this

  o means that the virus others and i have been trying to plant for
    a long time is slowly spreading

  o is very cheering

as you hint, it is far from evident how we can encourage the
cultural change to make this a core and operational value, as
opposed to phrases we say to appear sensible while we push for the
particular straws we feel are justified.  reducing others' features
is easy, it is our own which are hard to restrain.  but the union
of all those desires is, imiho, far too large.

and the iab and iesg get serious pushback for attempts to contain
the feature enthusiasm.  accusations of "not responsive,"
"controlling," and "undocumented process" are far easier to make
than it is to do simple and elegant design.  ( not to pretend that
the iab and iesg do not have communication and process problems )

we, as individuals, have to be willing to stand up and say "sorry,
but that makes me uneasy.  is it *really* necessary?  is it clearly
the simplest and best way to meet a real need?"

randy

---

From: avri doria <[email-masked]>
Date: Sun, 01 Dec 2002 14:34:24 +0100

>>general area issues on that list?

you are doing that, and i assume most of us are at least reading.

 > well, we're singin' duet here, but no one else is picking up the

well, it has been such an interesting duet, it is difficult to
know how to join in harmoniously.

my problem is i just don't think as quickly as you guys do.

> scaling chapter 42 paragraph 11

i worry about trying to apply the general notions of scaling to
this problem.  not because they might not have some relevance,
but because the scaling factors of organizations would be
radically different from the mechanistic notions of scaling
we normally deal with (not that i think we understand that realm
very well either, but that is grist for another mill)

so, while the meetings are good for sharing information, that
does not preclude them possibly also being helpful in getting to
know each other and our concerns.  but, or course, they can only
be one part of a a puzzle.  no one things represents the entire
problem, or the entire solution.

> and i suspect the book does not have a happy ending.  but then,

the ending really depends on the intentions of the cast of characters.
and i think that this is one of the critical differences between
human groups and groups of systems.  the intentions of the participants
makes much of the difference.  also the full complement of roles being
available in the group, and the relationships between those roles, makes
a difference.

i would argue that with cooperative intentions, even a very large
organization could work on the basis of trust without great gobs of
bureaucracy. and, as is mentioned below, a large bureaucracy can be
used to hobble an organization, if the intentions run that way.

i think that the interplay of individual interests with corporate
interests has injured the basis of trust.  and i think that the need
to keep a job and remain within the corporate fold, warps the normal
human interest in working cooperatively - i.e. of playing nicely in
the IETF sandbox.  i am not arguing that corporate interest can or
should be eliminated.  in fact my current speculation is that the IETF
fantasy that we are all independent agents  may actually be part of
the problem.  since  a person cannot differentiate when they speak for
themselves from when they speak for their masters, it forces many to
internalize their master's voice as their own.  this removes much of
their freedom of expression.  and while yes, there are some who speak
only in their own voices and not in a corporate voice, many of these
end up unemployed and running their own consultancies (yes , this is
a gross generalization based purely on anecdotal evidence).

so, i think one thing we need to understand in fixing our problems is
this interplay between personal expression and corporate expression.
i.e. this scared cow of the IETF must be examined, and perhaps fixed.

> ok.  i am pragmatic.  how about an ops chairs meeting in sf?  how

i think dinners are a good idea, but get togethers have to happen
with some degree of regularity.  the human dynamic that makes a
group work, needs time and continuity to form.

in fact, if these contacts are not continued and shown to have real
effect, they can become a new part of the problem breeding skepticism
and worse, bitterness.  so, in scheduling the first event, one must
have a commitment to continuity

>>a resolution and get back to the AD?

another thing i think is missing in this organization is a real
understanding of the roles within the leadership and the relationship
between these roles; e.g. between WG chairs in different areas.  i
personally think we have made an error in trying to model the AD-WG
chair on the corporate management model.  one aspect of this structure
is the organizational isolation of one areas chairs form another areas
chairs. a blatant effect of this organizational model is that we keep
searching for the role of the IAB in this hierarchy, and it doesn't
really exist - so we don't really know what to make of them.
in other words we have an inconsistent model for IETF organization that
does not help the individuals function in their roles.

>>Yes, I agree that seems to be what normally happens, and I suppose

i don't think it is necessarily so.  if the organizational model fits
the needs of the organization, including its identity, it can minimize
these risks.

but i am an optimist.

> produce quality relevant technology.

this i agree with.

a.

-- 
Avri Doria
http://www.sm.luth.se/~avri/

---

From: Avri Doria <[email-masked]>
Date: Sun, 01 Dec 2002 14:45:35 +0100

but i am an optimist.

this i agree with.

a.

---

From: Marshall Rose <[email-masked]>
Date: Mon, 2 Dec 2002 11:14:19 -0800

> the simplest and best way to meet a real need?"

randy - at the risk of speaking plainly: if the iesg would simply make
statements like this in a timely, open fashion (i.e., explain their reasoning),
then i doubt there would be any meaningful pushback.

a great deal of the perceived problems come from a lack of either timeliness
("let's sit on it until they go away") or less-than-openness ("let's say no, but
not tell them why").

whether or not this is the intention/agenda/etc. of the iesg is immaterial, in
both yokohama and atlanta, it was clear to me that this is how the folks in the
trenches are perceiving things...

/mtr

---

From: Randy Bush <[email-masked]>
Date: Mon, 02 Dec 2002 12:03:41 -0800

> Most WG members want WG chairs to be "above the fray." In
> particular, this means that they do not want WG chairs to have
> technical opinions about matters before the WG.

so we should choose them for lack of subject knowledge and look for
experience in the civil service and the united nations?

randy

---

From: Randy Bush <[email-masked]>
Date: Mon, 02 Dec 2002 12:20:51 -0800

> statements like this in a timely, open fashion (i.e., explain their
> reasoning), then i doubt there would be any meaningful pushback.
> 
> a great deal of the perceived problems come from a lack of either
> timeliness ("let's sit on it until they go away") or less-than-openness
> ("let's say no, but not tell them why").
> 
> whether or not this is the intention/agenda/etc. of the iesg is
> immaterial, in both yokohama and atlanta, it was clear to me that this is
> how the folks in the trenches are perceiving things...

you miss my point, marshall.  by 'individuals' i meant wg members,
not iesg members.

the point is that we have four choices

  o we all, as wg members, get a *lot* more diligent about
    producing clean, needed, and well-designed product, or
  
  o the iesg makes sure it is needed, cleans it up, and makes sure
    it is a good design, or

  o some or much of our product will not be really needed, clean,
    or well-designed, or
  
  o magic happens, sows' ears become silk purses, and cash falls
    from the sky :-)

i *far* prefer the first choice.  but, when it does not happen, i
have the thankless (and opprobrium attracting) job of the second.
and, being an engineer, i can not abide by the third or count on
the fourth.

randy

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2002 11:24:49 -0800 (PST)

>     producing clean, needed, and well-designed product, or

This one is mandatory.

>   o the iesg makes sure it is needed, cleans it up, and makes sure

If we add any more work to the existing IESG workload, we'll burn out IESG
members even fast, make it even harder to find new ones, and still not
solve the quality problem. I'd prefer to see an explicit goal to *reduce*
IESG workload significantly.

>   o some or much of our product will not be really needed, clean,

Vote for "not be really needed". We should be more hardcore about adding
WG work items (and WGs). In particular, I'd argue that work items or WGs
that involve liasons with other standards bodies are *really* expensive in
terms of IESG time -- and need to be thought through very carefully. There
may be a limit to the number of such relationships that can be
successfully managed at one time, given our resources.

>   o magic happens, sows' ears become silk purses, and cash falls

Cash can be made to appear if there is a use for it that would generate
cleaner, more timely and better designed protocols. However, I haven't
heard anyone point to lack of money as the culprit.

> i *far* prefer the first choice.

I do too, but some restraint in addition of WG work items (and WGs) is
probably also required.

---

From: "James Kempf" <[email-masked]>
Date: Mon, 2 Dec 2002 12:37:44 -0800

> experience in the civil service and the united nations?

I think we either need to specifically empower WG chairs to wade in and allow
them to pick and chose w.o. having to wield the IESG stick or learn to live with
the current situation where they pass through the technical quality problems and
let the IESG push back.

If we choose to do the former, then we need to make sure there are strict
controls in place such that any WG chair which abuses this power by supressing
other opinions is kept under control.

            jak

---

From: Frank Kastenholz <[email-masked]>
Date: Mon, 02 Dec 2002 16:08:37 -0500

>>     producing clean, needed, and well-designed product, or

how do you cause this to happen?
_some_ people can be educated.
but how do they deal with a working group
meeting with 500 people in it, 480 of whom
are in need of some architectural clue?
and how do they deal with the person who 
wants their Highschool Science Project
included in the final standard and will continue
to kick up a ruckus until it happens?

In other words, legislating intelligence is not
a good plan.

>IESG workload significantly.

First, in the long run, better output of WGs should reduce
the workload of the IESG (easier to make a small course correction
early on than a large one later), although there is an intermediate
time when they would have more work.

Second, the IESG can deputize directorati to go to meetings and
provide a fair amount of clue injection and then, if it fails
to take, call in the iesg-er of relevance to cause goodness.

Of course, if the iesger is more interested in keeping people
happy and making friends, this won't happen.

>WG work items (and WGs). 

The people guilty of this are working group chairs
and IESGers. How many of you have told your working
groups that some item of discussion on the mailing
list or a request for time at an IETF is not in order
at the current time? As opposed to the number that
send out notes to their working groups saying "We've
got an ietf meeting coming up, who wants to make a
presentation on their favorite highschool science
project?"

>In particular, I'd argue that work items or WGs

Having served on the IESG and not under NDA, I'll say that these
issues were not really big time sinks when I was on the
IESG (5 years ago), except for one or two ADs who live-breathe-
eat-sleep-and-defecate doing these sorts of things, but those ADs 
are also the kinds of people who get off on comparing how much 
the airlines kowtow to their super-deluxe-plutonium frequent flier 
card as compared to the rest of us, and who enjoy double-triple 
booking their time and always be running off to another meeting --
all to make us think they're very very important people... 

In other words, killing liasons  probably will not significantly
alter IESG workload.

Frank Kastenholz

---

From: Marshall Rose <[email-masked]>
Date: Mon, 2 Dec 2002 14:19:43 -0800

> you miss my point, marshall.  by 'individuals' i meant wg members,

well, you *may* be talking about WG chairs, but *I'm* talking about ADs. (so, i
don't think anything in your reply actually conflicts with what i said in my
email.)

consistent with the P-A-C-T stuff, i prefer surprises occuring earlier rather
than later, so yes, i too would prefer that WG chairs apply common sense both in
terms of process management and technical management.

>   o magic happens, sows' ears become silk purses, and cash falls

it's not really clear to me that the first two choices are in conflict.

/mtr

---

From: Erik Guttman <[email-masked]>
Date: Mon, 2 Dec 2002 23:26:07 +0100 (CET)

> In other words, legislating intelligence is not

I disagree with your assessment and interpretation, Frank.  We are not
in danger of being overrun by morons.  WG chairs can easily turn the
folks you mention away from the mike, or harder, shut them down from
jamming a mailing list.  

We have the opposite problem.  Those with critical intelligence are
dissuaded from involvement. 

Consider that
- WG discussion is often dominated by extroverts with terrible social
  skills.  
- Document authors are often more interested in publishing and
  defending their previous work than its technical quality.
- The active participants in WGs (the 3 to 8 of them) often become
  increasingly attached to the decisions they've made and generally
  become nasty if basic assumptions get challenged.

Folks rightly feel that the price of entry into the discussion is high
and the payback is low.  Most input will not make a difference to the
outcome.  To make deeply incisive comments is to ask for a fight. 

If there's anyone we should discourage from participation, its those
with extreme clue but minimal capacity to collaborate.  We need to get
to a point where people who are not in the core group feel they are
stakeholders in the process and that they will be heard.  Otherwise we
can just forget about getting any peer review for internet drafts.

How?

- The best way would be publishing only anonymous RFCs.  Most intransigent
  authors care a great deal about *their product* and much less about 
  listening, following instruction and consensus.  I am not saying do
  away with acknowledgements.  Just get rid of the author/editor credit.

- Put people on the spot, get them to volunteer to review documents,
  get them to arbitrate disputes.  Guilt and shame people into taking
  on work.  WG documents go to the IESG full of cruft just because
  none of the 4 active WG members could look at it again.

- Get our leadership to admonish folks to review docs early and often.
  When the IESG and IAB chair speak on the issue of participation and
  IETF conscience, people listen.  

- As WG chairs we must not tolerate ad hominem attacks, vapid arguments,
  off-topic drivel, repetitive rants or long winded posts.  Oops.

Erik

---

From: Melinda Shore <[email-masked]>
Date: Mon, 02 Dec 2002 17:50:51 -0500

> folks you mention away from the mike, or harder, shut them down from

As things stand, no we cannot.  The problem isn't obnoxious,
obstructive morons.  It's well-intentioned people with lousy
ideas who are able to convince other well-intentioned people
without the background to recognize lousy ideas.  It's very
easy for a working group to come to consensus on something
that's perfectly awful.  Right now, as working group chairs
we're not really given the latitude to shut that stuff down
by fiat.

>- The best way would be publishing only anonymous RFCs.

I've thought about that quite a bit.  In addition to the
problems you highlight it might discourage land rush
behavior when a new working group is formed.  At the same
time, it's hard enough to get volunteers to do work as it
is.  I'm not sure that it's true that dedicated smart people
would be just as willing to pitch in if they didn't get
their names on documents.

Melinda

---

From: Andy Bierman <[email-masked]>
Date: Mon, 02 Dec 2002 17:09:47 -0800

At 11:26 PM 12/2/2002 +0100, Erik Guttman wrote:
>>...

I agree with you to some extent, but I also think some of the blame
needs to fall on WG Chairs who don't have enough intestinal fortitude 
to make hard choices in their WG. These include:

  - forcing the WG to select between different proposals instead
    of letting the WG do all of them by merging documents
  - keeping the WG from over-engineering everything they touch
  - getting the WG to stick to the schedule in the charter

>outcome.  To make deeply incisive comments is to ask for a fight. 

This is the nature of standards work in the IETF. For better or worse,
we act more like the US Congress than we care to believe.  It's one
thing to introduce a bill (technical skills to produce a competent I-D)
but it's quite another to get a law signed by the President (political
skills to resolve conflict, affect compromise, survive dilution,
achieve vendor buy-in and deployment).

>  away with acknowledgements.  Just get rid of the author/editor credit.

I disagree strongly with this approach. Author credit is also
author accountability.  It's hard enough to get WG authors to
make frequent updates and address every detail and I-D nit as it is.
Maintaining technical quality and finishing on time will be much
harder to achieve if there's no author accountability.  

>  none of the 4 active WG members could look at it again.

How? It's one thing to assign responsibility to people, but what
if they don't do their homework?  It's quite common for these
IETF responsibilities to take a back seat to "day job" responsibilities.
It's tough to discipline a volunteer army.

>  IETF conscience, people listen. 

This is a good idea, but it should be handled when a WG starts a work item.
Some ADs are already doing this by assigning cross-domain technical
advisors. 

>  off-topic drivel, repetitive rants or long winded posts.  Oops.

Yes.  I think tools development is the next big step.  I know there
have been meetings on the use of Bugzilla to manage WG document progress,
but I couldn't attend those meetings.  Can someone post a pointer to
minutes, FAQ, usage examples, etc?  I would really like to get started
on using Bugzilla for the PSAMP WG.

>Erik

Andy

---

From: Ron Bonica <[email-masked]>
Date: Mon, 02 Dec 2002 23:22:11 -0500

Folks,

So far in this thread, folks have defined engineering discipline in terms of
the following rules:

1) Before you solve a problem, make sure that there is a real problem to be
solved
2) Make sure that your solution works
3) Make sure that your solution doesn't break anything else
4) Make sure that your solution isn't overly complex
5) Make sure that your solution doesn't preclude too many future solutions
to future problems

(I am sure that you folks can add a few more rules.)

In the best of all possible worlds, individual contributors impose this
discipline upon themselves. In the real world, that is populated by
well-meaning but sometimes clueless people, we need the IAB and IESG. The
alternative is architectural entropy.

Because the IAB and IESG are populated by extremely competent engineers,
they are very good at enforcing Rules 2 through 5. Rule 1 presents a harder
problem.

Identifying a real problem can be as much a matter of perspective than it is
a matter of engineering certainty. It is impossible to be 100% objective in
this regard. So, we might do well to be more tolerant of proposals that fall
into a gray area wrt Rule 1.

                                                                  Ron

---

From: Richard Shockey <[email-masked]>
Date: Mon, 02 Dec 2002 22:03:06 -0500

>we're not really given the latitude to shut that stuff down

I disagree... we've had lousy ideas in my WG ENUM ..and with the consensus
of my co-chair  they can be halted or at least redirected to a more useful
direction.

Every WG must have 2 chairs who are ultimately dedicated to the goal of
closing up shop.

At some point someone has to establish leadership and the buck stops IMHO
with the WG chair. What has disturbed me over time in the IETF is the lack
of WG chair leadership at the expense of attempting to attain consensus.

Put your foot down and and if the AD disagree then they can fire you ...but
stand for something for god's sake.

Personally I'd rather be chewed out by my  AD for being a hard ass and
pushing a WG to consensus or saying personally that I think a idea is
stupid than standing on the sidelines and seeing silliness prevail.

Half the problems the AD have is that most WG chairs try and be "nice guys"
and not force issues.

The IESG has the ultimate duty and obligation to halt stupid ideas as a
last resort.. indeed half the task of the IETF at this point is to stop
stupidity.

But we still let it happen ..OPES being a classic example.

Richard Shockey, Senior Manager, Strategic Technology Initiatives
NeuStar Inc.
46000 Center Oak Plaza  -   Sterling, VA  20166
Voice +1 571.434.5651 Cell : +1 314.503.0640,  Fax: +1 815.333.1237
<mailto:[email-masked]> or <mailto:[email-masked]>
  <http://www.neustar.biz> ; <http://www.enum.org>
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

---

From: Frank Kastenholz <[email-masked]>
Date: Tue, 03 Dec 2002 09:17:18 -0500

>folks you mention away from the mike, or harder, shut them down from

Sorry, I am guilty of using shorthand for people who have some
combination of the following traits
- Not completely dim
- Do not have _only_ the Internet's best interests at heart
- Have ideas or visions which are unsuitable to the internet
  'visitecture' (architecture > visitecture > vision-thing)
- Assume that their preconceptions are in fact how things are in 
  the internet, or how they should be
- Refuse to or are incapable of taking the time and sitting in the
  back of the room for a couple of years to learn how things are
  before trying to add to the pot,
- Refuse to or are incapable of understanding how things are, even
  if they do take the time...
- Believe that architecture, design, and analysis are things done
  by house-builders, dress-makers, and psychobabblists, respectively.
  real engineers Write Code...
but that'a a big mouthful and I can't type it in every time, so 
deal with it. I know it might hurt someone's feelings or damage
their self esteem, but I do not live in California and have no
desire to.

>dissuaded from involvement. 

This is true. Because the morons (see above diatribe) have taken 
over. Besides a higher than desired MFD (moron-field-density), 
there is another problem. Several years ago the IETF was 
chastised by some people for being too rude and obnoxious.
We were perceived as being "too American". So now we've become
very polite and sip our tea from fine china cups with our
little fingers sticking up in the air and all that. The problem
is, many of the smarter, capable, people with "critical
intelligence" loudly proclaim bad ideas for what they are -- 
stupid. But we can't call things stupid, it's impolite, it's
too American, it's not PC, it damages people's self esteem.
And we can't have that. So the smart, American, un-PC, self-
esteem-damagers went off to do other things.

>  away with acknowledgements.  Just get rid of the author/editor credit.

No it wouldn't. Many intransigent authors still will claim
ownership. Also, for many, the only place where the claim 
of ownership is important is the yearly review when they go
to their manager when they get brownie points for publishing
documents. The other class don't care about credit at all, so
long as the right ideas get pushed forward. Companies have agendas.
They send people to the IETF to get those agendas promoted.
The authorship is not important, getting the right things advanced
is. If my box has a swoopy new feature and my competitor doesn't,
I want to get that feature standardized so I get a competitive
advantage. Yes, we all recite the mantra that the IETF is composed
of individuals, not companies. But it's a load of bullshit.

>  none of the 4 active WG members could look at it again.

How do we arbitrate things? Who does it? What happens when the
lawyers come in? There's way too much money on the table for
things to be a nice happy little village where everyone does
the right thing...  (even if they could agree on the right thing)

>  IETF conscience, people listen.

Been tried in various ways for various things at various times.
The results were not encouraging.

Frank Grumpenholz

---

From: Margaret Wasserman <[email-masked]>
Date: Tue, 03 Dec 2002 09:48:40 -0500

>to facilitate the process and nothing more.

Unfortunately, this doesn't match my opinion, or the the stated
expectations of most of the IESG members that I work with, regarding
the role of WG chairs...

I think that it is the job of the WG chairs to ensure that the documents
that are submitted to the IESG are both technically sound and useful.
This needs to involve management of the process -- making sure that
documents receive sufficient review, etc.  But, it also require the WG
chair to execute judgement about when documents are ready to be sent
to the IESG, and to block sending documents that have serious technical
flaws or are not generally useful.

Of course, our current process has a bit of problem there...  The WG
chairs can hold document while their technical flaws are corrected, or
until their usefulness is demonstrated, but they can't stop the
authors from submitting the same documents as individual submissions,
if they become impatient with the WG process.

>without a major change in the expectations of WG members about the chair's 

Then, maybe that major change is needed...

>either they agree or the WG documents don't got to RFC.

I think that this sort of appeal to IESG authority has led to some of the
current perception problems in the IETF regarding the role of the IESG.

>any technical opinion except theirs to be discussed.

There is a difference between shutting down discussion on an alternative,
and ensuring the quality of the end result.  In fact, lively discussion of
alternatives is more likley to lead to a quality result.

Margaret

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 3 Dec 2002 07:42:17 -0800

Hi -

> From: "Andy Bierman" <[email-masked]>
> To: "Erik Guttman" <[email-masked]>
> Cc: "Frank Kastenholz" <[email-masked]>; "Bernard Aboba"
<[email-masked]>; "Randy Bush"
> <[email-masked]>; "Marshall Rose" <[email-masked]>;
<[email-masked]>
> Sent: Monday, December 02, 2002 5:09 PM
> Subject: Re: engineering discipline WAS: stats from the text conferencing
experiment
...
...

The IETF puts far more faith in the technical judgment and writing
skills of document editors than some other organizations.  The kind
of "Keith understands the edits" editors' instructions we rely on would
be unthinkable.  By requiring editors to do much more than
mechanically apply editing instructions agreed by the working group,
we impose a significantly heavier burden on editors than for standards
of similar complexity in other organizations.  (In fairness, our choice
of tools (nroff) makes mechanical editing easier here than in organizations
where editors have to bludgeon Microsoft Word into submission.)  This
over-reliance on the skills of the document editors also has the effect of
giving WG members an excuse for not reviewing documents closely.

The part that I've found to be a real pain is dealing with document
editors who do not deliver, who fail to participate, or work at
cross-purposes to WG consensus.  Just at a people-management
level, it is *much* easier to make staff changes when the editors'
names aren't on the cover page of the work.

Randy Presuhn

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 03 Dec 2002 17:26:58 +0100

> expectations of most of the IESG members that I work with, regarding

Nor mine. On the contrary, I think WG members want chairs to be fair-minded
and even-handed, but they absolutely *must* have the technical depth
and non-moron intelligence to be smarter about the subject matter than
most other people in the room. (Which is a tall order BTW.)

They also need a lot of technical breadth, because actually the chairs 
are the first line of defence against ideas that look good at first
sight but happen to be broken. If the chairs spot this, there are
ways of making the idea go away (like no agenda time, no draft-ietf-*
filename, and "outside charter" closedowns on email threads). If the
chairs let a broken idea through, it develops a life of its own.

   Brian

---

From: "James Kempf" <[email-masked]>
Date: Tue, 3 Dec 2002 09:39:46 -0800

> most other people in the room. (Which is a tall order BTW.)

The difficulty is that sometimes, there are ideas that come up in which the
technical flaws may be subtle or the ideas may be flawed for other reasons. An
example might be fit with work going on in another working group. Clearly, there
should be an interest in keeping the IP protocol suite as lean as possible, but
working group momentum and turf protection might lead to a working group pushing
on some work items duplicating that in other working groups. Another example,
gathering evidence that a particular protocol or idea which the chair or even an
AD intuitively feels is bad might be extremely time consuming, requiring
simulations, implementations, perhaps extending to a multiple year effort. And
as the IETF takes up more and more complex areas, I think such cases may become
more common.

These are cases where a certain amount of engineering judgement is required, and
if the chair or even an AD is pushing for one particular proposal based on their
engineering judgement, it may be hard to resist the combined arguments of WG
members. In some cases, it seems on the surface as if it is just a question of
the WG chair's opinion v.s. that of the WG. The easy cases are simple for the WG
chairs to shoot down.

> chairs let a broken idea through, it develops a life of its own.

Too true. :-(

            jak

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Tue, 3 Dec 2002 12:22:37 -0700

Advocacy is for the working group members; chairmanship for dealing
fairly with the conflicting advocates.  

The overall contempt that many here have expressed for IETF participants
is a problem that is evident in public and drives many good, non-combative
people far away from the organization.

Don't confuse small-mindedness with wisdom.

Hilarie

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 Dec 2002 11:17:38 +0100

"The Purple Streak, Hilarie Orman" wrote:
...
> Don't confuse small-mindedness with wisdom.

I don't think it's small-minded to observe that we quite often
get people with great persistence and foolish or discredited
ideas showing up in the IETF. I think we should be free to deal
fairly quickly with these. OTOH we should always be open to people
who are proposing unconventional ideas or who are clearly in fast-learning
mode.

  Brian

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 3 Dec 2002 13:01:03 -0500

Sometimes a WG Chair or AD or IESG has approached the IAB about having
an IAB member (or set of IAB members) inject some architectural advice
into a WG
on a particular topic.  And sometimes that works.  The thing to keep in
mind
is that the IAB is truly powerless to force anyone do anything (and I
consider
that to be a feature :-).

	However, sometimes an IAB person can persuade a WG to change its mind
by the presentation or publication of a well-reasoned
architectural/technical
analysis, particularly if made by someone who hasn't been involved in
any WG fray
up until that point.  The real trick is having a solid
architectural/technical
presentation, of course.  And the IAB has no monopoly on clues, so one
can
use the same approach with other normal IETF folks who can be persuaded
to
document/present their rationale.

	And IAB members are volunteers also, with lots of demands on their
time, so this is an approach best used selectively rather than
frequently.
However, this is a potential tool that is in the community's toolbox.

Ran
[email-masked]

---

From: "James Kempf" <[email-masked]>
Date: Tue, 3 Dec 2002 08:53:48 -0800

> harder to achieve if there's no author accountability.

This certainly hasn't been the case with other standards groups that don't have
explicitly authored specifications. The technical quality of IEEE specifications
tends to be high, and that's also the case for 3GPP specifications, regardless
of one's opinion on 3GPP's underlying architecture. Getting rid of authored
specifications would also remove the incentive for companies to require authored
drafts as part of people's performance evaluations, and the tendency in some
quarters to view IETF meetings as equivalent to technical conferences, where the
authors give research papers.

That said, the RFC series is really set up to be much like an authored
publication process, and changing this underlying assumption would be a big
deal.

            jak

---

From: RJ Atkinson <[email-masked]>
Date: Wed, 4 Dec 2002 13:41:37 -0500

> publication process, and changing this underlying assumption would be

I've got a lot of sympathy for the above position.

An intermediate position would be that in all cases of IETF
standards-track
documents, there is no "author" but instead an "editor" (implying the
AUTHOR
contact information in the document is retitled as EDITOR information
and that
the cover page also explicitly notes that John Doe or Jane Doe is
EDITOR.  This
is done sometimes already.  This does not fix the "employers measure
the number
of authored/edited IETF documents in performance appraisals" issue.

Ran

---

From: Andy Bierman <[email-masked]>
Date: Wed, 04 Dec 2002 11:02:37 -0800

>authors give research papers.

I should clarify my statement above:
- I could support anonymous RFC authorship, but first I would want to fully
  understand the ramifications of such a big change. I suspect it will cause
  as many problems as it attempts to solve. This change alone will not
  fix the IETF.
- I understand anonymous authorship has worked for other standards bodies,
  but they do not pretend (as we do) that the WG mailing list is the best
  mechanism for creating technical standards.  Instead, they rely on lots
  of face-to-face meetings to make progress.  Decisions are more likely to
  be the result of a committee effort instead of the work of one or two authors.

<rant>
In my experience in the IETF, no significant standard has ever been completed
without at least one multi-day interim meeting. IMO, the easiest way to improve
quality and timeliness is to acknowledge the importance of these face to face
meetings.  The current IETF meeting format (e.g., 2 hours * 3 times a year)
is a joke.  The only useful purpose they serve is as a document revision deadline.
</rant>

Andy

---

From: Pat Calhoun <[email-masked]>
Date: 04 Dec 2002 16:34:08 -0800

> is a joke.  The only useful purpose they serve is as a document revision deadline.
I have to agree. Let's face it, once people leave the IETF meeting, they
get busy, and there is rarely a pressing need (or pressure) to get work
completed before the next meeting. Having regular interim meetings (ala
802) would certainly help in shortening the process.

oh, btw, I also suspect that attendance at interim meetings would also
be limited to people that really care.

PatC

---

From: Jonathan Rosenberg <[email-masked]>
Date: Wed, 04 Dec 2002 23:30:36 -0500

> 802) would certainly help in shortening the process.

I agree as well. In fact, I have noticed an increasing trend in my 
groups to use the IETF itself as the "interim meeting". That is, more 
time at IETF is spent in design team meetings than in the sessions 
themselves. After all, everyone is at the IETF, what better time to hold 
meetings for the various design teams?

-Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                72 Eagle Rock Ave.
Chief Scientist                             First Floor
dynamicsoft                                 East Hanover, NJ 07936
[email-masked]                     FAX:   (973) 952-5050
http://www.jdrosen.net                      PHONE: (973) 952-5000
http://www.dynamicsoft.com

---

From: "Lisa Dusseault" <[email-masked]>
Date: Thu, 5 Dec 2002 10:55:30 -0800

I've seen this too (interim design meetings at IETF meetings) and I
consider them as harmful if not handled carefully.

If "design team" meetings are regularly scheduled at the same time as
other working group meetings, participants are actively discouraged from
gaining breadth of knowledge by attending related working groups.
Participants may attend the IETF for only a day or two, go to their
primary working group, go to their design team meetings, have no time
left over for anything else and go home.  Later,
interoperability/integration issues come up as a surprise.

Chairs have a responsibility (on top of all other responsibilities) to
encourage their WG members to acquire breadth as well as depth of
knowledge on potentially related working groups, specifications, and
technology.  Of course that may conflict with what many see as the
chair's primary responsibility, that of getting a draft to RFC in any
form whatever.

Lisa

> -----Original Message-----
> From: Jonathan Rosenberg [mailto:[email-masked]]
> Sent: Wednesday, December 04, 2002 8:31 PM
> To: Pat Calhoun
> Cc: Andy Bierman; James Kempf; Erik Guttman; Frank Kastenholz; Bernard
> Aboba; Randy Bush; Marshall Rose; [email-masked]
> >>without at least one multi-day interim meeting. IMO, the easiest way
to
> improve
> >>quality and timeliness is to acknowledge the importance of these
face to
> face
> >>meetings.  The current IETF meeting format (e.g., 2 hours * 3 times
a
> year)
> >>is a joke.  The only useful purpose they serve is as a document
revision
> deadline.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Thu, 05 Dec 2002 14:25:24 -0500

> gaining breadth of knowledge by attending related working groups.

In the meetings I have been involved in, the design team meetings were 
scheduled to avoid any sessions that the team members wanted to go to. 
Other meetings often occur at night or over meals. Most folks in a 
design team typically go to the same set of working groups, so its 
really not that hard to find an open slot.

Of course, this does eliminate the ability to attend groups which are 
truly outside of your area of work (groups you might attend if you had 
nothing else to do). However, if you've not really been actively 
following the list and reading the drafts, you are not likely to gain 
much benefit from the meeting anyway. I personally seldom have the 
chance to attend groups really outside my general area of interest, much 
as I would like to. There are so many groups just doing voip-related 
work that I cannot even attend all of those (avt, enum, ieprep, iptel, 
megaco, midcom, mmusic, rohc, sigtran, sip, sipping, speechsc, sprits, 
impp, simple).

I suspect that this is true for many other people as well. I wonder how 
much cross fertilization there REALLY is these days between normally 
disparate groups. It would be an interesting measurement to make - poll 
people about what groups they attend, and ask which of those relate to 
their primary area of work, and which were attended just to get breadth 
of knowledge.

-Jonathan R.

---

From: Aaron Falk <[email-masked]>
Date: Fri, 6 Dec 2002 13:44:35 -0800

Margaret Wasserman wrote:
> >technical opinions about matters before the WG. Basically, the
> >current popular model of WG chairs is that they are there to
> >facilitate the process and nothing more.
> 

same here.  The wg chair has a difficult job in that they have to
balance consensus and clue injection which can sometimes be in
conflict.  I've drawn on my AD for 'moral support' in the past when an
unpopular clue was not welcome.

> chairs can hold document while their technical flaws are corrected,
> or until their usefulness is demonstrated, but they can't stop the
> authors from submitting the same documents as individual
> submissions, if they become impatient with the WG process.

disagree: This is why the RFC Editor circulates individual submissions
to the IESG: to prevent an end-around to the existing wg process.

--aaron

---

From: Margaret Wasserman <[email-masked]>
Date: Fri, 06 Dec 2002 18:22:13 -0500

>to the IESG: to prevent an end-around to the existing wg process.

Sorry, I guess I was a bit unclear...  The WG chair can't hold a
document alone, so he would do best to make sure that his AD is
on board before using this type of tactic.  Otherwise, the
document is more likely to come around the other way.

Margaret

---

From: avri doria <[email-masked]>
Date: Thu, 05 Dec 2002 21:10:12 +0100

> their primary area of work, and which were attended just to get breadth

I too think this would be interesting.  It think it marks one of the
changes in the IETF in the last decade.  I remember conversations a long
time ago that basically would chastise the participant who only attended
the WGs they were involved in because they limited cross fertilization.
  We then went through the phase a few years ago where 'over attendance'
caused us to object to tourist attendees.  We made all kinds of
draconian pronouncements about who was allowed to sit in the front rows,
and castigated, by implication at least, those attendees who had not
done all of their homework.  The rules against presentations also limit
the usefulness of WG attendance for cross fertilization purposes.

a.