Subject: [Eligibility-discuss] we only need to count RFC/WG-draft authors

From: Michael Richardson <[email-masked]>
Date: Wed, 1 Jul 2020 19:09:47 -0400

On 2020-06-28 1:47 p.m., Michael Richardson wrote:
> 
> Hi, Robert ran some numbers, and some venn diagrams were produced based upon
> some additional questions. I can't find the post with the details, but I did
> print the diagrams, which I hope will get shared by those who still have
> them.  Brian has had some adventures turning this into SVG for inclusion in
> his draft, and I expect the diagrams will make it to the list very soon.
> 
> So I'm writing this down from a paper copy :-)
> 
> EXEC SUM: I like rule 5, and I would expand it to adopted WG draft authors.

Brian has posted the Venn diagrams that my review was based upon, and 
has been hacking SVG into an internet draft to show the results.
This thread went astray into issues of how many are on the 2020 list.

I propose that we ask Robert/Stephen/Brian to delete path 2 (WG Chair) 
and 4 (IESG) from the criteria, and add path 6: is an active author of a 
WG adopted I-D.  It's probably a different code path from 5, so might as 
well list it separately.
(Question: did path 5 include ISE authors?)

I have nothing against path 2 (WG Chairs), I just doubt that it catches 
anyone additional.

I think that path 4 (IESG/IAB member) has bad optics. And it seems 
really likely that it catches nobody.

I don't have the raw numbers for this.

> (1-2018-5yr) of the 779 previously eligible,
>      379 were also eligible according to either criteria, of which 142 volunteered.
>      An additional 43 volunteers were eligible only by 3-of-5.
>      357 3-5 eligible people did not volunteer.
>      Conclusion 2018/5yr brings 542 potential volunteers to the pool.
> 
> (2-2018-3yr) of the 779 previously eligible,
>      379 were also eligible according to either criteria, of which 138 volunteered.
>      An additional 47 volunteers were eligible only by 3-of-5.
>      353 3-5 eligible people did not volunteer.
>      Conclusion 2018/3yr brings 364 potential volunteers to the pool.
> 
> (3-2019-5yr) of the 797 previously eligible,
>      361 were also eligible according to either criteria, of which 115 volunteered.
>      An additional 56 volunteers were eligible only by 3-of-5.
>      380 3-5 eligible people did not volunteer.
>      Conclusion 2018/5yr brings 484 potential volunteers to the pool.
> 
> (3-2019-3yr) of the 797 previously eligible,
>      334 were also eligible according to either criteria, of which 113 volunteered.
>      An additional 58 volunteers were eligible only by 3-of-5.
>      405 3-5 eligible people did not volunteer.
>      Conclusion 2018/3yr brings 303 potential volunteers to the pool.
> 
> High-level conclusions: it is not surprising that going back 2 additional
> years brings more people to the 2,4,5 path.  In 2018, 4 of the additional
> people were already eligible by 3-of-5, and in 2019, that number is 25.
> 
> I would like to split out the 2 (WGchairs), 4 and 5 categories.
> a) I'm unclear in the data analysis if category 4(IESG/IAB) properly excluded
>     currently sitting members.  It might be worth changing criteria (4) to say
>     that all outgoing I* from the past 3 years are eligible.  But, really
>     anyone who has been on the I* in the past two years is going to 3-of-5
>     eligible anyway.  I think that category (4) is not a useful thing.
> 
> b) I would like to know how many WG chairs are category (2) eligible, while
>     not already being 3-of-5 eligible.  I could imagine 20% are below the
>     line.  Having WG chairs on the nomcom is especially important, because
>     they have the most experience with the current leadership.  Having WG
>     chairs on the nomcom is a problem, because it's precisely from that
>     pool of people that the nomcom winds up having to select from.
>     That is, it benefits nomcom little to add people to the volunteer pool who
>     will decline to serve because they wish to be considered.
> 
> c) I conclude that adding WG draft authors to the list provides a significant
>     and useful increase to the pool.  The sum on each case is around 1000
>     (958,1136,1110,929), which is of the same order of magnitude as the number
>     of people who attend meetings.
>     Noting that these are *WG* RFC authors, and not just anyone who
>     can get a datatracker account, I think that the criteria is good.
>

---

From: Ted Lemon <[email-masked]>
Date: Wed, 1 Jul 2020 19:11:36 -0400

On Jul 1, 2020, at 7:09 PM, Michael Richardson <[email-masked]> wrote:
> I propose that we ask Robert/Stephen/Brian to delete path 2 (WG Chair) and 4 (IESG) from the criteria, and add path 6: is an active author of a WG adopted I-D. It's probably a different code path from 5, so might as well list it separately.

But I thought we had a policy of not valorizing authorship. Hence the five author rule.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 01 Jul 2020 19:56:13 -0400

> But I thought we had a policy of not valorizing authorship. Hence the five author rule.

yes, that's definitely a concern, and you are absoluting right to ask this.

And a reason I wanted a structured way to do acknowledgments :-)
I have no problem with paths that involve reviewing documents, being
secretary, being document shepherd, etc.

Do you agree that the path 2 (WG Chair) and path 4 (IESG) gets us little?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Ted Lemon <[email-masked]>
Date: Wed, 1 Jul 2020 19:58:15 -0400

I do, yes. In fact, I would go so far as to say that this would probably have the result of perpetuating dysfunction where it exists. 

> On Jul 1, 2020, at 19:56, Michael Richardson <[email-masked]> wrote:
>

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 00:01:13 +0000

I think our problem is too much inbreeding. I would rather see something like "once you've been a WG chair, IESG, IAB member you are ineligible to serve on the next nomcom after you leave your position."

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 1 Jul 2020 20:08:36 -0400

I would like to keep the WG Chairing qualification in the list.  And 
under any reasonable definition of "knows the community" having been on 
the IESG or IAB would seem to qualify.  (Whether being on the trust, or 
being an IAB appointed RSSAC person, or ... should qualify is a 
different and harder question.)

I do realize that those rules are likely to capture few people.  But 
they seem to represent an important aspect of our intent.

Yours,
Joel

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 1 Jul 2020 20:11:27 -0400

That is a particularly strange rule proposal (ineligible after you leave 
WG chairing) since WG chairs are eligible to serve on nomcom under 
current rules.

The usual premise I thought we as a community had agreed on was that we 
were trying to approximate "has a reasoanble understand of what the 
community is and some perspective on what it needs."  We are not trying 
to select for people we happen to agree with.  Or, I guess, for people 
we happen to disagree with.  or for people who specifically agree with 
or disagree with the incumbents.

Yours,
Joel

---

From: Ted Lemon <[email-masked]>
Date: Wed, 1 Jul 2020 21:02:38 -0400

The thing is, our rules have always been a poor proxy for what we really want. What we really want (imho) is for ietf /contributors/ to be on nomcom. Attendees are simply a proxy for contributors, because deciding who has contributed is difficult to do without being subjective, and that leads to a problem similar to valorizing authorship or leadership. 

> On Jul 1, 2020, at 20:11, Joel M. Halpern <[email-masked]> wrote:
> 
> That is a particularly strange rule proposal (ineligible after you leave WG chairing) since WG chairs are eligible to serve on nomcom under current rules.
> 
> The usual premise I thought we as a community had agreed on was that we were trying to approximate "has a reasoanble understand of what the community is and some perspective on what it needs."  We are not trying to select for people we happen to agree with.  Or, I guess, for people we happen to disagree with.  or for people who specifically agree with or disagree with the incumbents.
> 
> Yours,
> Joel
> 
> Eligibility-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/eligibility-discuss

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 1 Jul 2020 20:15:26 -0500

On Wed, Jul 1, 2020 at 6:56 PM Michael Richardson <[email-masked]>
wrote:

>     > On Jul 1, 2020, at 7:09 PM, Michael Richardson <
> [email-masked]>
>     > wrote:
>

Chuckle. In a parallel discussion on gendispatch, I just replied to a
concern about long working group draft author lists often left untrimmed
until after publication is requested. It's worth deciding whether we are
more worried about having too many authors on working group drafts, or too
few authors :D

Best,

Spencer

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 01:42:36 +0000

Well, I was being terse.  I would assume that current WG chairs, current I* members, would not be on nomcom, and I was proposing a one-year lockout in addition.

    were trying to approximate "has a reasoanble understand of what the 
    community is and some perspective on what it needs." 

Sure.

>  We are not trying 

I don't see how I was saying that.

I have heard that we want IETF leadership to be more representatives, and that a more representative nomcom is a requirement for that. Which to me means more diversity, and less of the "same old" membership pool.

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 02 Jul 2020 11:47:17 +1000

On Thu, Jul 2, 2020, at 11:42, Salz, Rich wrote:
> Well, I was being terse.  I would assume that current WG chairs, 
> current I* members, would not be on nomcom, and I was proposing a 
> one-year lockout in addition.

(Not saying anything about your proposal, except perhaps to note that it is orthogonal to this discussion.)

Excluding chairs from NomCom would disqualify a great many people.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 01:49:03 +0000

>    (Not saying anything about your proposal, except perhaps to note that it is orthogonal to this discussion.)

Yes, I should have started a separate thread.

>    Excluding chairs from NomCom would disqualify a great many people.

It would be interesting to see what the actual counts have been.

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 2 Jul 2020 02:52:47 +0100

Hiya,

On 02/07/2020 02:42, Salz, Rich wrote:
> I have heard that we want IETF leadership to be more representatives,
> and that a more representative nomcom is a requirement for that.
> Which to me means more diversity, and less of the "same old"
> membership pool.
More diversity: great.
Less "same old": great.
Representation of any kind: bad plan for the IETF.

I think the ultimate problem with asking for representation
in an IETF context, is that representation requires us to
know the "registered voter list" (or equivalent) which is
fairly antagonistic to our stated goal of only requiring an
email address (and maybe a sensible opinion) to allow one
influence the Internet (to the extent we have such
influence). "Representation" would require us to have
paid-up/voting members.

Having nomcom as a subset of IETF participants look like
a near-random selection of IETF participants is a good
think. Confusing that with representation seems like an
error.

Cheers,
S.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 1 Jul 2020 21:57:19 -0400

We do exclude I* serving members, on the basis that the leadership 
should not be selecting the leadership.
However, we have not, and I think should not, extended that to WG 
chairs.  On the one hand, the WG chairs are the folks who need to work 
with the leadership, and so have a lot of perspective on what is and is 
not working.
And, excluding all WG chairs would drastically trim the eligible pool.

Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 2 Jul 2020 14:49:24 +1200

> Well, I was being terse.  I would assume that current WG chairs, current I* members, would not be on nomcom, and I was proposing a one-year lockout in addition.

Current I* members are disqualified already; locking them out for a an extra year might marginally reduce the risk of self-perpetuation, but I think it's slight. Most I* members want a respite year after they step down anyway.

As Martin said, excluding WG Chairs would have a pretty large impact on the pool size, and furthermore who is better placed to judge IESG performance than a WG Chair?

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 2 Jul 2020 14:54:44 +1200

> I don't have the raw numbers for this.

FWIW, I don't plan to update the draft again before the cutoff (2020-07-13),
especially if it involves rerunning the data analysis & recreating the
figures. Also, it's above my pay grade to call consensus on any proposed
changes to the paths.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Wed, 01 Jul 2020 23:01:15 -0400

Ted Lemon <[email-masked]> wrote:
    > I do, yes. In fact, I would go so far as to say that this would
    > probably have the result of perpetuating dysfunction where it exists.

I don't think it will actually perpetuate, because I think that I* and WG
chairs already all qualify by 3-of-5.

BUT, I do think that it definitely creates a perception that we want to
perpetuate dysfunction.

Salz, Rich <rsalz=[email-masked]> wrote:
    > I think our problem is too much inbreeding. I would rather see
    > something like "once you've been a WG chair, IESG, IAB member you are
    > ineligible to serve on the next nomcom after you leave your position."

Sitting WG chairs can be on the nomcom now, and it's very important to have
them.   But, I agree with I* members ought to take a break. I think most do.

---

From: Ted Lemon <[email-masked]>
Date: Wed, 1 Jul 2020 23:09:27 -0400

This wasn’t a problem when we were doing three in person meetings a year. Without that, suddenly it becomes a problem. I would not be surprised if we can’t meet again in person until next July. So that really emphasizes all of the leadership eligibility paths. 

> On Jul 1, 2020, at 23:01, Michael Richardson <[email-masked]> wrote:
>

---

From: S Moonesamy <[email-masked]>
Date: Thu, 02 Jul 2020 02:54:01 -0700

Hi Rich,
At 05:01 PM 01-07-2020, Salz, Rich wrote:
>something like "once you've been a WG chair, IESG, IAB member you 
>are ineligible to serve on the next nomcom after you leave your position."

"Inbreeding" is generally not good for an organization as people do 
not live forever.  The Nomination Committee, in its current form, is 
expected to be representative of the "community", or specifically, a 
subset of the "community" which comprises people who understand IETF 
processes.  If the IETF does not bring in new people, it will end up 
facing that problem.

I took a quick look at the current list of volunteers.  Approximately 
6% of the persons on the list are of female gender.  14% of IESG 
members are of female gender.  I doubt that the process is biased 
against persons of female gender.  The point here is that it is 
useful to consider whether the Nomination Committee, as a whole, 
could be unfairly prejudiced for or against one or more candidates.

I don't think that your idea is bad as it is useful to consider all 
options.  It does not necessarily have to be a rule unless it is to 
address a problem which the Nomination Committee encountered.

I came across the following report: 
https://datatracker.ietf.org/meeting/98/materials/slides-98-ietf-sessc-nomcom-report-00 
One of the areas of weaknesses which was identified is that some 
areas has a very small leadership pool.  Is that a side effect of 
"inbreeding"?  I would say that it could be that.

I mentioned leadership in previous paragraph.  That word is sometimes 
interchanged with "management".  It could be said that there is 
leadership if IETF Management can address the small pool problem.

Regards,
S. Moonesamy

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 2 Jul 2020 07:30:25 -0500

Hi, Rich,

> It would be interesting to see what the actual counts have been.

I'm not counting, but as an approximation, we've had something like 100
working groups for 15-20 years, and the number of working groups with one
chair is probably less than the number with three chairs. So if we say we
just disqualified 200 chairs, that's worth noting ...

I tripped over https://datatracker.ietf.org/wg/1wg-summary.txt several
years ago, which has all of the active working groups listed with their
chairs, in one place, if looking at names helps.

Best,

Spencer

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 13:18:10 +0000

*   I'm not counting, but as an approximation, we've had something like 100 working groups for 15-20 years, and the number of working groups with one chair is probably less than the number with three chairs. So if we say we just disqualified 200 chairs, that's worth noting ...

(Thanks for changing the thread subject)

That’s not accurate enough, since we don’t know how many WG chairs check the “willing to server” box, as opposed to saying “I’m a WG chair, that’s enough service” or “I’m a WG chair and want more, so going to try for I*”  And as a sanity-check, more than 200 for the nomcom pool is a reach.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 13:22:17 +0000

Perhaps it's just an Americanism, but there is a difference between saying something is "representative" (the second quote above), contrasted wth saying people are "reprentives" of a larger group (the first quote above).

We want a "representative body" meaning it looks like the general IETF population.  Not we want a body of representatives.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 13:30:22 +0000

>    And, excluding all WG chairs would drastically trim the eligible pool.

Some people think that, but we don't know that.  See my response in the (proper) subthread.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 2 Jul 2020 08:43:53 -0500

Hi, Rich,

On Thu, Jul 2, 2020 at 8:18 AM Salz, Rich <[email-masked]> wrote:

>
>    - I'm not counting, but as an approximation, we've had something like
>    100 working groups for 15-20 years, and the number of working groups with
>    one chair is probably less than the number with three chairs. So if we say
>    we just disqualified 200 chairs, that's worth noting ...
>
>
>
> That’s not accurate enough, since we don’t know how many WG chairs check
> the “willing to server” box, as opposed to saying “I’m a WG chair, that’s
> enough service” or “I’m a WG chair and want more, so going to try for I*”
> And as a sanity-check, more than 200 for the nomcom pool is a reach.
>

Sure. I think my point should have been that all the names on that list
would be out, whether they would have volunteered or not.

The volunteer pool is (as of today) about 160, but the
eligible-to-volunteer pool tends to be about 700-800, isn't it?

(Sitting in a teleconference, so not checking now - my apologies)

Best,

Spencer

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 13:48:10 +0000

*   Sure. I think my point should have been that all the names on that list would be out, whether they would have volunteered or not.

Okay, that’s more accurate.  I am not sure if it is significant, since other threads are talking about ways to increase the pool size. But even if it is, we have to think about things systemically and not just look at one factor, as I am positive you will gree.

  *   (Sitting in a teleconference, so not checking now - my apologies)

Hope you’ve got a nice background :)

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 2 Jul 2020 08:50:21 -0500

Hi, Rich -

On Thu, Jul 2, 2020 at 8:48 AM Salz, Rich <[email-masked]> wrote:

>
>    - Sure. I think my point should have been that all the names on that
>    list would be out, whether they would have volunteered or not.
>
>
>
> Okay, that’s more accurate.  I am not sure if it is significant, since
> other threads are talking about ways to increase the pool size. But even if
> it is, we have to think about things systemically and not just look at one
> factor, as I am positive you will gree.
>

Yup. Systematic is good.

>
>    - (Sitting in a teleconference, so not checking now - my apologies)
>
>
>

Sadly, we're in GoToMeeting, which I don't believe has the same cool
background support as Zoom :-)

Best,

Spencer

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 2 Jul 2020 09:59:52 -0400

Rich, the eligible pool, as used in e.g. Brian's analysis, is not 
limited by those who are willing to serve.  It is the seet of people who 
could serve.

As such, even if we note that some people chair more than 1 working 
group, excluding WG chairs would reduce the eligible pool by between one 
hundred and two hundred people.  That is a large portion of the pool.

And, as separately noted, the WG chairs have the best visibility into 
whether the leadership is doing their job right.

Do you  want, by transitivity, to also exclude the WG secretaries and 
the document shepherds appointed by the WG chairs?  That is the logical 
extension of your argument.

Yours,
Joel

On 7/2/2020 9:18 AM, Salz, Rich wrote:
>   * I'm not counting, but as an approximation, we've had something like
>     100 working groups for 15-20 years, and the number of working groups
>     with one chair is probably less than the number with three chairs.
>     So if we say we just disqualified 200 chairs, that's worth noting ... 
> 
> the “willing to server” box, as opposed to saying “I’m a WG chair, 
> that’s enough service” or “I’m a WG chair and want more, so going to try 
> for I*”  And as a sanity-check, more than 200 for the nomcom pool is a 
> reach.
> 
>

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 14:16:03 +0000

Yes, thanks, I know.

I am concerned about those who actually volunteer.  So while it reduces the potential pool, our statistics seem to indicate that the ACTUAL pool isn't affected very much. And we don't know if saying "nobody in power can pick the next generation of leaders" will result in more folks participating. There are enough liaisons that I don't believe we're at risk of losing all institutional knowledge.

I don't know about that; one could argue that stepping away gives a broader perspective. Or that being *on* the leadership gives ultimate insight as to what is working, or not, and yet we decided the trade-offs for that situation aren't worth it.

I didn't say that, and I am a bit surprised that you are trying this reduction ad absurdium approach.

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 2 Jul 2020 15:23:09 +0100

FWIW, I'm on a call too 😊

Looking at Barbara's latest pool of 143 volunteers, I see 42 (I think, I'm doing this from memory) who are WG chairs. That seems, to me, to be pretty drastic.

Best
Adrian

-----Original Message-----
From: Eligibility-discuss <[email-masked]> On Behalf Of Salz, Rich
Sent: 02 July 2020 15:16
To: Joel M. Halpern <[email-masked]>
Cc: [email-masked]
Subject: Re: [Eligibility-discuss] Impact of excluding chairs

Yes, thanks, I know.

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 14:37:37 +0000

>    Looking at Barbara's latest pool of 143 volunteers, I see 42 (I think, I'm doing this from memory) who are WG chairs. That seems, to me, to be pretty drastic.

Yes it is significant.  Thanks for counting them.  It isn't clear that the trade-off is still not worth making, considering the overall context.

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 2 Jul 2020 15:49:27 +0100

Hiya,

On 02/07/2020 15:37, Salz, Rich wrote:
>> Looking at Barbara's latest pool of 143 volunteers, I see 42 (I
>> think, I'm doing this from memory) who are WG chairs. That seems,
>> to me, to be pretty drastic.
> 
> Yes it is significant.  Thanks for counting them.  It isn't clear
> that the trade-off is still not worth making, considering the overall
> context.

I think excluding WG chairs is an interesting idea and like
the idea of improving diversity, but on balance I figure I'd
likely be against excluding WG chairs as I think nomcom would
lose to much process-knowledge and history as a result and
could end up making worse decisions.

Cheers,
S.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 02 Jul 2020 10:51:56 -0400

Ted Lemon <[email-masked]> wrote:
    > This wasn’t a problem when we were doing three in person meetings a
    > year. Without that, suddenly it becomes a problem. I would not be
    > surprised if we can’t meet again in person until next July. So that
    > really emphasizes all of the leadership eligibility paths.

Okay, this is a good point, and I admit that I forgot that part of the reason that
we were tweaking this :-)

As such, I would be happy to keep path 2 (WG/BOF Chairs).

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 11:00:02 -0400

That’s good, but we probably need to think bigger. Trying to bandaid this isn’t likely to work well. Document authors are more good than bad. It would be really good to be able to somehow capture other contributions though. 

> On Jul 2, 2020, at 10:51, Michael Richardson <[email-masked]> wrote:
>

---

From: Suresh Krishnan <[email-masked]>
Date: Thu, 2 Jul 2020 15:17:12 +0000

> On Jul 2, 2020, at 10:49 AM, Stephen Farrell <[email-masked]> wrote:
> 
> 
> Hiya,
> 

I am with Joel and Stephen here. I think disenfranchising WG chairs here would be a really bad idea. In addition to the process knowledge and history mentioned above, WG chairs might be in the best position to evaluate I* performance.

Thanks
Suresh

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 15:35:25 +0000

>    I am with Joel and Stephen here. I think disenfranchising WG chairs here would be a really bad idea. In addition to the process knowledge and history mentioned above, WG chairs might be in the b…

I rough glance at the last four nomcom memberhips show 2-3 WG chairs out of 10.  That is measurable, but not clear if it's significant. There is much institutional knowledge through the one-third non-voting members.

What kind of "best position to evaluate" are you thinking of?  Current membership or something else? Does it only affect incumbency? Is it something that can't be equally provided via sending in feedback?

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 2 Jul 2020 11:36:44 -0400

>
> I am with Joel and Stephen here. I think disenfranchising WG chairs here
> would be a really bad idea. In addition to the process knowledge and
> history mentioned above, WG chairs might be in the best position to
> evaluate I* performance.
>

+1 to that - excluding WG chairs is crazy talk!

Cheers,
Andy

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 15:37:57 +0000

*   +1 to that - excluding WG chairs is crazy talk!

I think we should measure and consider impacts, rather than just pejoratively dismiss things.

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 2 Jul 2020 11:53:52 -0400

Rich,

I guess I deserve that :-), but others have already made the case better
than I could why WG chairs should be on NomCom. I certainly do agree with
them.

Cheers,
Andy

On Thu, Jul 2, 2020 at 11:38 AM Salz, Rich <[email-masked]> wrote:

>
>    - +1 to that - excluding WG chairs is crazy talk!
>
>

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 16:04:30 +0000

> but others have already made the case better than I could why WG chairs should be on NomCom

To me it isn’t clear that the case has been made. We know that the potential pool could be diminished, we have a datapoint that shows an actual pool being diminished, and rough statistics showing nomcom membership would change by a couple of members. But that is only considering “no WG chairs” *in isolation.*

We don’t know if others would sign up.  We could add a “WGchairs liaison” if we felt it necessary. We don’t know if there would be any benefit by forcing nomcom to be more representative of more of the IETF by drawing it only from non-current leadership. We don’t know how important being a current WG chair is to the evaluation process.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 02 Jul 2020 13:15:44 -0400

> eligible-to-volunteer pool tends to be about 700-800, isn't it?

Yes, that's correct.
779 in 2018 and and 797 in 2019.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 02 Jul 2020 13:20:14 -0400

An additional consideration is that IESG ADs can only really on be selected
from among those who are WG chairs.<debate this elsewhere>

One way in which a WG chair gets a feeling for whether or not they can/want
to make it through the nomcom process is by being on the nomcom :-)

Obviously, volunteering to serve is no guarantee of it, but the experience
can be somewhat transitive, so one WG chair can tell others about the
experience.

---

From: "Pete Resnick" <[email-masked]>
Date: Thu, 02 Jul 2020 13:30:59 -0500

> chairs already all qualify by 3-of-5.

We've had the occasional remote chair. I think it wouldn't be bad to 
include them. I agree that I* is probably a no-op, and I understand the 
perception problem, so perhaps a 1-year hiatus is useful, but I don't 
feel strongly about it.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 18:48:11 +0000

Most of the last few Security AD's haven't been.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 02 Jul 2020 14:48:12 -0400

Ted Lemon <[email-masked]> wrote:
    > That’s good, but we probably need to think bigger. Trying to bandaid
    > this isn’t likely to work well. Document authors are more good than
    > bad. It would be really good to be able to somehow capture other
    > contributions though.

Do we have to get there in one step?
Can we do this incrementally?
The community might not have the energy for that, because each time we want
to add something, we wind up having to argue the problem statement again.

It could be that we could agree on a palette of five or eight mechanisms, and
a timeline to RFC3999 experiment them, and a process for the community to
decide to keep or abandon them.
But, the timescale for such an experiment is probably 10 years.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 14:54:39 -0400

> to add something, we wind up having to argue the problem statement again.

I don’t think we can get there in one step, but we should at least have an idea in mind for how to get there so that we can take more than one step.  I think noticing who is a participant is something that we need to learn how to do _before_ we start deciding how to use the output of that statistics collection effort.

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 2 Jul 2020 13:21:26 -0700

> Most of the last few Security AD's haven't been.

All of the last four ADs had been chair at one time

Me: TLS
Roman: DOTS
Kaduk: Kitten
Kathleen: MILE

I'm too lazy to determine whether people were chairs at the time they
became AD.

-Ekr

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 20:23:30 +0000

*   All of the last four ADs had been chair at one time

I meant “at the time.”  Perhaps I was focusing too much on “who are WG chairs” as opposed to “who have been WG chairs.”

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 2 Jul 2020 13:28:07 -0700

On Thu, Jul 2, 2020 at 1:23 PM Salz, Rich <[email-masked]> wrote:

>    - All of the last four ADs had been chair at one time
>
>
>
> I meant “at the time.”  Perhaps I was focusing too much on “who are WG
> chairs” as opposed to “who have been WG chairs.”
>

OK.  I believe Kaduk and Roman were both chairs when they were made AD. I
was not, and if I am reading the history of MILE correctly, Kathleen was.

-Ekr

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 2 Jul 2020 20:41:51 +0000

*   OK.  I believe Kaduk and Roman were both chairs when they were made AD. I was not, and if I am reading the history of MILE correctly, Kathleen was.

Thanks for the correction.  My take-away is numbers matter :)

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 2 Jul 2020 13:49:29 -0700

> was not, and if I am reading the history of MILE correctly, Kathleen was.

Amusingly enough, we were both chairs of ACE when made AD.

I will dutifully encourage Jim and Daniel to put their names in this time
around, but will also be encouraging a broader candidate pool :)

-Ben

---

From: Michael Richardson <[email-masked]>
Date: Thu, 02 Jul 2020 17:23:22 -0400

> I don’t think we can get there in one step, but we should at least have
    > an idea in mind for how to get there so that we can take more than one
    > step.  I think noticing who is a participant is something that we need
    > to learn how to do _before_ we start deciding how to use the output of
    > that statistics collection effort.

While I like this concept/process a lot, I don't know how to put this in
practice.

I like the idea of detailing the outcome we want, and then finding technical
measures to get there.  But... "Noticing who is a participants" is easy to
say and maybe very hard to get consensus on.
I'm reminded of Phaedrus' quandry on Quality.
   https://en.wikipedia.org/wiki/Pirsig%27s_Metaphysics_of_Quality

But, maybe I'm overthinking this.  How would you proceed?

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 17:30:10 -0400

On Jul 2, 2020, at 5:23 PM, Michael Richardson <[email-masked]> wrote:
>   https://en.wikipedia.org/wiki/Pirsig%27s_Metaphysics_of_Quality <https://en.wikipedia.org/wiki/Pirsig%27s_Metaphysics_of_Quality>
> 

The most obvious way is to use the datatracker to formally request reviews from working group participants at various points in the process. Anyone who does a review contributed. If someone does a review that is utterly useless, we might have a feedback process for saying “this was crap” but we don’t actually do that for people who sit in the room, so even without that I think this is a useful measure. Abusive reviews can be handled the way other abusive behavior is in the IETF—we already have a process for that.

The way we do reviews and discussion in working groups right now really sucks—there’s no formal process, nobody is keeping track of what’s being discussed, and it’s much more costly than it ought to be because of that.  I don’t know if we can entirely fix that, but it wouldn’t hurt to start taking steps in that direction. Formal reviews in the datatracker is a pretty obvious step that would require minimal additional tooling. I can think of others to try after this one has been tested.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 2 Jul 2020 17:37:24 -0400

Your description of "the way we do reviews" seems to miss many elements.
SOme working groups do assign reviewers.  Those that do generally track 
that the reviewers are provided, and responded to.

The many directorates do provide reviews which are assigned, tracked, 
recorded, and acted upon.  With tooling support.

Yours,
Joel

On 7/2/2020 5:30 PM, Ted Lemon wrote:
> On Jul 2, 2020, at 5:23 PM, Michael Richardson <[email-masked] 
> <mailto:[email-masked]>> wrote:
> The most obvious way is to use the datatracker to formally request 
> reviews from working group participants at various points in the 
> process. Anyone who does a review contributed. If someone does a review 
> that is utterly useless, we might have a feedback process for saying 
> “this was crap” but we don’t actually do that for people who sit in the 
> room, so even without that I think this is a useful measure. Abusive 
> reviews can be handled the way other abusive behavior is in the IETF—we 
> already have a process for that.
> 
> The way we do reviews and discussion in working groups right now really 
> sucks—there’s no formal process, nobody is keeping track of what’s being 
> discussed, and it’s much more costly than it ought to be because of 
> that.  I don’t know if we can entirely fix that, but it wouldn’t hurt to 
> start taking steps in that direction. Formal reviews in the datatracker 
> is a pretty obvious step that would require minimal additional tooling. 
> I can think of others to try after this one has been tested.
> 
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 17:47:39 -0400

On Jul 2, 2020, at 5:37 PM, Joel M. Halpern <[email-masked]> wrote:
> SOme working groups do assign reviewers.  Those that do generally track that the reviewers are provided, and responded to.

If “the way we do reviews” is different from WG to WG, then what I described can’t work.

Also, I don’t know of WGs that track reviews using the datatracker.  Is this already an available feature?

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 2 Jul 2020 17:53:11 -0400

The WGs that do formal reviews that I know of track them on a wiki page. 
  At a guess that is because they have not felt they need the tooling 
that is available.  Since the need for reviews, and the value of them, 
and the stages they are most useful at, seems to vary from WG to WG, I 
would not want to see us making strict rules calling for such.

Many of the directorates use the tooling for assigning, tracking, etc 
their reviews.

Yours,
Joel

On 7/2/2020 5:47 PM, Ted Lemon wrote:
> On Jul 2, 2020, at 5:37 PM, Joel M. Halpern <[email-masked] 
> <mailto:[email-masked]>> wrote:
>> SOme working groups do assign reviewers.  Those that do generally 
>> track that the reviewers are provided, and responded to.
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 18:09:52 -0400

On Jul 2, 2020, at 5:53 PM, Joel M. Halpern <[email-masked]> wrote:
> The WGs that do formal reviews that I know of track them on a wiki page.  At a guess that is because they have not felt they need the tooling that is available. Since the need for reviews, and the value of them, and the stages they are most useful at, seems to vary from WG to WG, I would not want to see us making strict rules calling for such.

Any particular reason why not?

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 2 Jul 2020 18:58:52 -0400

The reason for not writing strict rules is that, as I said, different 
situations are different.   Within a working group, the kinds of reviews 
needed, the stages they are needed at, and even the number that are 
useful varies widely.  I would like more ideas to get actual working 
group engagement, rather than formal procedures.  (And I acknowledge 
that even that is because of specific working groups and probably does 
not apply to all WGs.)

And making up WG procedures for the purpose of creating eligibility 
criteria seems backwards to me.  And our focus on this list is 
eligibility criteria.

Yours,
Joel

On 7/2/2020 6:09 PM, Ted Lemon wrote:
> On Jul 2, 2020, at 5:53 PM, Joel M. Halpern <[email-masked] 
>> The WGs that do formal reviews that I know of track them on a wiki 
>> page.  At a guess that is because they have not felt they need the 
>> tooling that is available. Since the need for reviews, and the value 
>> of them, and the stages they are most useful at, seems to vary from WG 
>> to WG, I would not want to see us making strict rules calling for such.
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 19:07:17 -0400

On Jul 2, 2020, at 6:58 PM, Joel M. Halpern <[email-masked]> wrote:
> The reason for not writing strict rules is that, as I said, different situations are different.   Within a working group, the kinds of reviews needed, the stages they are needed at, and even the number that are useful varies widely.  I would like more ideas to get actual working group engagement, rather than formal procedures.  (And I acknowledge that even that is because of specific working groups and probably does not apply to all WGs.)

I think you’re really overcomplicating this. The datatracker is a tool for tracking who is doing reviews. Anybody can do a review. There doesn’t need to be a process for this. If you don’t care about getting “credit” for doing reviews, there’s no need to do a review in the tracker. If you do care, then you should do it in the tracker. Working groups should socialize this as a way to get credit for contributing for non-authors. No?

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 11:29:02 +1200

> Can we do this incrementally?

That's really why the draft proposes a process experiment with a 2 year lifetime. Hopefully by the end of that we will have (a) experience and (b) a stable new normal.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 11:31:11 +1200

Yo! Let's not debate the IETF process itself here. The eligibility rules
have to assume something, and assuming today's reality seems best.

Regards
   Brian

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 19:45:28 -0400

> have to assume something, and assuming today's reality seems best.

Brian, I get that you want to not discuss this, but you don’t have the right to unilaterally declare discussions out of scope, and honestly it’s pretty frustrating.  We do not have a way to enfranchise everyone who is a regular IETF participant but not a regular draft author. That is a PROBLEM. One that we’ve been able to successfully ignore for a long time.  It’s always been the case that lots of IETF participants can’t attend in person 3/5 times.  What has changed recently is that that number may soon become _all IETF participants_ instead of _lots of IETF participants_. We’ve kicked this can down the road every time I’ve brought it up over the past decade. This is not okay.

This is eligibility-discuss. We are discussing an eligibility issue. Please stop trying to shut it down like this.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 11:54:31 +1200

> 779 in 2018 and and 797 in 2019.

If we asked Robert Sparks nicely, he might be able to actually answer
the question of how much excluding chairs would reduce the eligible pool.

fwiw, my opinion is that we should include the chairs, as people well
qualified to evaluate IESG/IAB performance.

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 12:02:41 +1200

On 03-Jul-20 11:45, Ted Lemon wrote:
> On Jul 2, 2020, at 7:31 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:

It's a valid debate. But changing how the IETF does reviews is a long-term project that IMHO deserves its own draft and its own discussion.

Also, we did have 

>    o  Path 3: Has submitted at least 6 reviews as a member of an
>       official IETF review team within the last 3 years.

in the draft originally, and took it out because of earlier discussions of the same issues.

   Brian

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 20:13:38 -0400

> It's a valid debate. But changing how the IETF does reviews is a long-term project that IMHO deserves its own draft and its own discussion.

Why is this a “long-term project”?

It sounds like it’s a never-going-to-do project. It’s just not that hard, Brian.

>>      official IETF review team within the last 3 years.

Right. I’m saying that we should add to that formal working group reviews of documents.  Where formal just means “if you want to formally review the document so that there’s a record that you did, here’s how.”  This is not a heavy lift.

> in the draft originally, and took it out because of earlier discussions of the same issues.

I honestly don’t get that. If someone is on a directorate and is doing reviews, they are contributing. If their reviews suck, the AD(s) responsible for that directorate should give feedback. But they still contributed.  This is not a heavy lift.

The earlier discussions on this issue derailed on being perfect: only reviews that were “good” were tp count as contributions, because otherwise we’d be encouraging useless reviews.  Well, yeah, but we already have that problem.  If that’s a problem, we can add a process for dealing with it, but best is the enemy of good enough.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 13:46:53 +1200

OK Ted, why don't you work with Robert Sparks to figure out the code to
list the people who would qualify under the former Criterion 3 plus
official WG reviews?

I'm not being snarky. If we can do that, it's a candidate criterion that
the Secretariat could apply when requested by next year's NomCom Chair.
If we can't do that, it's not a usable criterion.

Regards
   Brian Carpenter

On 03-Jul-20 12:13, Ted Lemon wrote:
> On Jul 2, 2020, at 8:02 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 22:01:32 -0400

I think it would be helpful if you could stop throwing down gauntlets. This could be done in a code sprint, sure, but we don’t have to write code before we can reason about whether we want to add reviews as a way to qualify. 

> On Jul 2, 2020, at 21:46, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 22:02:48 -0400

To amplify on that, because you said you weren’t just being snarky: the data tracker knows who did a review. So we can write that code. 

> On Jul 2, 2020, at 22:01, Ted Lemon <[email-masked]> wrote:
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 2 Jul 2020 21:43:54 -0500

(top posting)

I was confused between "reviews as part of a review team that are assigned
and recorded in the datatracker, that review team secretaries know about"
and "reviews performed while a draft is still in the working group, by
working group participants".

Neither criteria is WRONG, but the first category of reviews is already in
the datatracker, and (for the review teams I've worked with) the dataset is
usually in good shape. The second category was what got us into the weeds
of varying practice between working groups (and, likely, areas), and isn't
a dataset that we can easily lay our hands on in the near term, so that we
can use it for eligibility next year, with confidence that someone has been
watching it.

So I think this thread is slouching toward analysis of the first category,
but not the second. Is that what other people are thinking?

Best,

Spencer

On Thu, Jul 2, 2020 at 9:02 PM Ted Lemon <[email-masked]> wrote:

> To amplify on that, because you said you weren’t just being snarky: the
> data tracker knows who did a review. So we can write that code.
>
> > I think it would be helpful if you could stop throwing down gauntlets.
> This could be done in a code sprint, sure, but we don’t have to write code
> before we can reason about whether we want to add reviews as a way to
> qualify.
> >
> >>>>> On Jul 2, 2020, at 8:02 PM, Brian E Carpenter <
> [email-masked] <mailto:[email-masked]>> wrote:
> >>>>> It's a valid debate. But changing how the IETF does reviews is a
> long-term project that IMHO deserves its own draft and its own discussion.
> >>>
> >>> Right. I’m saying that we should add to that formal working group
> reviews of documents.  Where formal just means “if you want to formally
> review the document so that there’s a record that you did, here’s how.”
> This is not a heavy lift.
> >>>
> >>> I honestly don’t get that. If someone is on a directorate and is doing
> reviews, they are contributing. If their reviews suck, the AD(s)
> responsible for that directorate should give feedback. But they still
> contributed.  This is not a heavy lift.
> >>>
> >>> The earlier discussions on this issue derailed on being perfect: only
> reviews that were “good” were tp count as contributions, because otherwise
> we’d be encouraging useless reviews.  Well, yeah, but we already have that
> problem.  If that’s a problem, we can add a process for dealing with it,
> but best is the enemy of good enough.
> >>>
> >>
>
> --

---

From: Ted Lemon <[email-masked]>
Date: Thu, 2 Jul 2020 22:51:19 -0400

What I proposed way back at the beginning of this tangent was that we put a system in place for tracking working group document reviews, like we currently track directorate reviews, so that two years from now when we are all sheltering in place from the dragon pox pandemic, we are able to include more ietf contributors than we would be able to include now. 

Separately, I think counting directorate reviews is pretty sensible. 

> On Jul 2, 2020, at 22:44, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> 
> (top posting)
> 
> I was confused between "reviews as part of a review team that are assigned and recorded in the datatracker, that review team secretaries know about" and "reviews performed while a draft is still in the working group, by working group participants". 
> 
> Neither criteria is WRONG, but the first category of reviews is already in the datatracker, and (for the review teams I've worked with) the dataset is usually in good shape. The second category was what got us into the weeds of varying practice between working groups (and, likely, areas), and isn't a dataset that we can easily lay our hands on in the near term, so that we can use it for eligibility next year, with confidence that someone has been watching it.
> 
> So I think this thread is slouching toward analysis of the first category, but not the second. Is that what other people are thinking?
> 
> Best,
> 
> Spencer
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 3 Jul 2020 15:17:03 +1200

> To amplify on that, because you said you weren’t just being snarky: the data tracker knows who did a review. So we can write that code. 

For the official review teams, yes. Not for reviews done within the WG, unless I'm mistaken. That's why I drafted Criterion 3 the way it was.

   Brian

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 3 Jul 2020 06:36:13 -0500

Hi, Ted,

On Thu, Jul 2, 2020, 21:51 Ted Lemon <[email-masked]> wrote:

> What I proposed way back at the beginning of this tangent was that we put
> a system in place for tracking working group document reviews, like we
> currently track directorate reviews, so that two years from now when we are
> all sheltering in place from the dragon pox pandemic, we are able to
> include more ietf contributors than we would be able to include now.
>

For this draft at this point in time, I suggest that we treat this as an
orthogonal point that is pretty sensible, but requires discussion/work
before we can add it to the eligibility criteria - among other things, it's
not just about eligibility, and could be worth doing whether it is ever
included in the Nomcom eligibility criteria or not.

We're just over a month from IETF 108, or I'd suggest bringing this up at
gendispatch. Do people consider the best way forward to be discussion on a
new mailing list, or do we have an existing mailing list that could host
development of proposals for better tracking of reviews done in a working
group?

I'm also adding "dragon pox pandemic" to my 2022 bingo card, but that's
also a seperate issue!

Best,

Spencer, who is fully expecting the return of the dinosaurs in November,
during IETF 109

>> >>> I honestly don’t get that. If someone is on a directorate and is
>> doing reviews, they are contributing. If their reviews suck, the AD(s)

---

From: Ted Lemon <[email-masked]>
Date: Fri, 3 Jul 2020 09:44:46 -0400

On Jul 3, 2020, at 7:36 AM, Spencer Dawkins at IETF <[email-masked]> wrote:
> Spencer, who is fully expecting the return of the dinosaurs in November, during IETF 109 

I think I’ve been insulted!  :)

> For this draft at this point in time, I suggest that we treat this as an orthogonal point that is pretty sensible, but requires discussion/work before we can add it to the eligibility criteria - among other things, it's not just about eligibility, and could be worth doing whether it is ever included in the Nomcom eligibility criteria or not.

We were having a discussion about how to improve the pool. I agree that we aren’t ready to codify “working group reviews” as a way to do it, but I am frustrated with the notion that we need to form a new mailing list to talk about it. Is the concern that this is so controversial that it will delay the work? Or that we have such short attention spans that if we allow discussion of this here, nobody will work on the draft, and the whole process will be derailed? I’m having trouble coming up with a threat model that I believe in.

> We're just over a month from IETF 108, or I'd suggest bringing this up at gendispatch. Do people consider the best way forward to be discussion on a new mailing list, or do we have an existing mailing list that could host development of proposals for better tracking of reviews done in a working group?

Specifically, this is a way to prevent discussion from occurring. You’re making a bunch of extra work. Why? I’m not saying you’re wrong, but it sounds like either you think the people who need to discuss this aren’t here, and that’s why we shouldn’t discuss it here, or you think some other thing sort of like that.

What I am proposing is something that we could simply put forth as a process experiment and see if there is uptake. Authors of drafts don’t need to do this; people who contribute but are not authors might want to do it. If nobody does, then maybe it’s not worth doing. There’s some cost to doing the experiment, but I don’t think it needs to be as big a deal as seems to be being made about it here.

BTW, does being a co-author on an IRTF RG draft get you nomcom eligibility?  :)

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Fri, 3 Jul 2020 08:57:10 -0500

> I think I’ve been insulted!  :)

Not at all. :D

> We were having a discussion about how to improve the pool. I agree that we
> aren’t ready to codify “working group reviews” as a way to do it, but I am
> frustrated with the notion that we need to form a new mailing list to talk
> about it. Is the concern that this is so controversial that it will delay
> the work? Or that we have such short attention spans that if we allow
> discussion of this here, nobody will work on the draft, and the whole
> process will be derailed? I’m having trouble coming up with a threat model
> that I believe in.
>

My concern (and some days, I don't even speak for me) is that this mailing
list targets discussion on Nomcom eligibility, and likely doesn't include
people who are interested in improving review handling, but don't know that
we're having that conversation here. I don't actually care how someone lets
those people know (a note to gendispatch or the IETF discussion list saying
that we're talking about improving review handling could be sufficient.

> Specifically, this is a way to prevent discussion from occurring. You’re
> making a bunch of extra work. Why? I’m not saying you’re wrong, but it
> sounds like either you think the people who need to discuss this aren’t
> here, and that’s why we shouldn’t discuss it here, or you think some other
> thing sort of like that.
>

That's exactly my concern.

> What I am proposing is something that we could simply put forth as a
> process experiment and see if there is uptake. Authors of drafts don’t need
> to do this; people who contribute but are not authors might want to do it.
> If nobody does, then maybe it’s not worth doing. There’s some cost to doing
> the experiment, but I don’t think it needs to be as big a deal as seems to
> be being made about it here.
>

Sounds like there's an opportunity to propose a process experiment.
Excellent.

> BTW, does being a co-author on an IRTF RG draft get you nomcom

Thank you for asking that question. Nomcom does review IAB membership (who
provide oversight of the IRTF and who select the IRTF Chair), so that seems
like an important thing to mention, no matter what the answer is.

Best,

Spencer

---

From: Ted Lemon <[email-masked]>
Date: Fri, 3 Jul 2020 10:15:25 -0400

On Jul 3, 2020, at 9:57 AM, Spencer Dawkins at IETF <[email-masked]> wrote:
> My concern (and some days, I don't even speak for me) is that this mailing list targets discussion on Nomcom eligibility, and likely doesn't include people who are interested in improving review handling, but don't know that we're having that conversation here. I don't actually care how someone lets those people know (a note to gendispatch or the IETF discussion list saying that we're talking about improving review handling could be sufficient. 

Okay, that makes sense, but it’s not what I was actually trying to do.

What I was trying to do was ask the question, “would it be a good idea to figure out a way to count IETF participation other than leadership, physical attendance, document authorship and directorate participation?” I think we should answer that question “yes,” and if we do, then indeed there is work to be done, but so far I keep getting a response of “we don’t want to talk about that, go do the work elsewhere” which seems wrong-headed to me.

Perhaps everybody reading that assumes that the answer is “yes” and that I am proposing to do all the work to implement that here, but I’m really just asking the question: do we think that other IETF participation, specifically reviewing documents in working groups, should count towards nomcom eligibility?

---

From: Michael Richardson <[email-masked]>
Date: Fri, 03 Jul 2020 13:03:51 -0400

> Brian, I get that you want to not discuss this, but you don’t have the
    > right to unilaterally declare discussions out of scope, and honestly
    > it’s pretty frustrating.  We do not have a way to enfranchise everyone
    > who is a regular IETF participant but not a regular draft author.

Ted, you *appear* to be suggesting changes to IETF process, yet I don't think you
really intend to do that.

I think that you are suggesting formalizing some existing and common IETF
process, ideally in a way that we can mechanically measure.
The challenge is (as Joel points) that there are many variations in the
process now, and making it measureable will constrain the variations.

So, if I may restate your goal:
   The goal is to enfranchise people who contribute to documents without
   requiring them to be authors.

This was in the back of my head when I posted this in 2019:
   https://mailarchive.ietf.org/arch/msg/rfc-interest/PLlExd7KrFRqiXszJKBmD3cdFOo/

    > That is a PROBLEM. One that we’ve been able to successfully ignore for a
    > long time.  It’s always been the case that lots of IETF participants
    > can’t attend in person 3/5 times.  What has changed recently is that
    > that number may soon become _all IETF participants_ instead of _lots of
    > IETF participants_. We’ve kicked this can down the road every time I’ve
    > brought it up over the past decade. This is not okay.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 03 Jul 2020 13:07:21 -0400

> Separately, I think counting directorate reviews is pretty sensible.

I suggest that we trivially have consensus that this is a good thing, and
that we know that we can trivially measure this in the datatracker, and
we should add this as a path.

{How we expand this, is a new problem}

---

From: Michael Richardson <[email-masked]>
Date: Fri, 03 Jul 2020 13:11:04 -0400

> That's really why the draft proposes a process experiment with a 2 year
    > lifetime. Hopefully by the end of that we will have (a) experience and
    > (b) a stable new normal.

I'm suggesting that just the initial set of criteria is not enough.

That we need to suggest/detail some additional criteria and a timeline to deploy.
That we not change too many things at once, but that we know what the
choices.

That might take a new IETF Consensus process, perhaps represented in a new
trivial document, but I think that doing this would allow the entire
community to see that we are thinking about many things, but that we just
want to do certain things first.

That way, proponents of method X (which might take more DT work), see that we
are not discounting that, but just not making it stage 1.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael Richardson <[email-masked]>
Date: Fri, 03 Jul 2020 16:24:51 -0400

> {How we expand this, is a new problem}

I recognize that this was in fact path (3) that was removed.

I think that it was removed hastily, and maybe because of the specific
numbers, rather than because we hated it as a concept.

I also remember comments about people doing crappy reviews just to get
counted.

This is why I want a structed way to count acknowledgements, because
generally, useful reviews get clearly acknowledged.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 4 Jul 2020 08:26:28 +1200

> want to do certain things first.

Exactly right. Time tends to run quickly. The call for NomCom volunteers for
2021-22 will go out in May 2021, so we have 10 months to put the Secretariat
in a position where they can run a script to qualify or disqualify people.
We could of course decide to include directorate reviews on that timescale,
but I simply don't believe (based on the IETF's normal rate of progress
on process improvements) that we could make it on time for WG-internal
reviews.

We could also, if we do this as a formal RFC 3933 process experiment,
write into the draft the ability for the IESG to add some criteria for
the 2022-23 NomCom, such as counting WG-internal reviews.

> are not discounting that, but just not making it stage 1.

Yes, certainly. I'm all for flexibility as the experiment continues.

(I also realise that the draft lacks a section on how to evaluate the
process experiment and what happens afterwards.)

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 4 Jul 2020 08:39:02 +1200

On 04-Jul-20 05:07, Michael Richardson wrote:
> I suggest that we trivially have consensus that this is a good thing, 

That's a bit disconcerting, because after the first round of discussion,
Stephen and I had the exact opposite impression. Just for clarity, the

>>       official IETF review team within the last 3 years.

and the feedback (around the end of March) was that this was too easy
to game, so we took it out in version -02 of the draft. If the rough
consensus now is to put it back in, I won't complain, although it would
need a new round of data analysis.

> and

Indeed. See my previous note. Again, I'm not against in principle, I
just see a practical problem.
 
  Brian

---

From: Ted Lemon <[email-masked]>
Date: Fri, 3 Jul 2020 16:44:08 -0400

Y’all may not be aware that there have been cases where someone who had been doing useless reviews consistently would try to get draft authors to add them to the acknowledgments. So this sort of abuse has already occurred in the past. The good news is that someone who would do this is unlikely to be effective on nomcom. And if there is a reward for getting acknowledged, that will make people more resistant to acknowledging bad reviewers just to get them to shut up. 

To some extent I think we worry too much about abuse. The existing criteria also created opportunities for abuse. I would like to think that this has not completely corrupted nomcom thus far. 

So indeed, counting acknowledgements could be an easy approach to this. We would need to design an etiquette for reminding draft authors to acknowledge when appropriate though, whereas using the data tracker takes that burden from the authors. In some cases an author will not want to acknowledge a negative review, even when it was actually beneficial. 

> On Jul 3, 2020, at 16:25, Michael Richardson <[email-masked]> wrote:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 4 Jul 2020 09:01:16 +1200

> Y’all may not be aware that there have been cases where someone who had been doing useless reviews consistently would try to get draft authors to add them to the acknowledgments. So this sort of ab…

All the same, getting a uniform standard for what deserves an acknowledgment seems unlikely to me. I've tended to err on the side of politeness - if someone finds a typo on page 43, at least it shows that they've read the whole thing, even if their contribution is non-technical. On the other side of the balance, I could probably find 50 RFCs where I've made a substantive comment and received no
acknowledgment. (Not complaining, just saying.)

Speaking of "contribution", if something counts as a contribution under BCP 79, isn't the contributor showing active participation in the IETF? Should that make them eligible?

Are minute-takers and jabber scribes eligible?
 
> To some extent I think we worry too much about abuse. The existing criteria also created opportunities for abuse. I would like to think that this has not completely corrupted nomcom thus far.

Right, but by adding on-line actions to the mix, we do create a risk of sock puppets managing to qualify themselves, so we have to be a little attentive to that.

    Brian

---

From: Ted Lemon <[email-masked]>
Date: Fri, 3 Jul 2020 17:52:51 -0400

> Are minute-takers and jabber scribes eligible?

I think that BCP 79 contribution is actually what should trigger eligibility. It’s a slightly higher bar than mere attendance. 

The cases I mentioned where people tried to game acknowledgments were all cases where the person didn’t attend in person often enough to qualify that way. But I think they were detected and handled reasonably well. I don’t think that’s a serious risk. I’m actually more concerned about someone who deserves acknowledgment not getting it.

---

From: Michael Richardson <[email-masked]>
Date: Sat, 04 Jul 2020 19:23:16 -0400

Ted Lemon <[email-masked]> wrote:
    > The cases I mentioned where people tried to game acknowledgments were
    > all cases where the person didn’t attend in person often enough to
    > qualify that way. But I think they were detected and handled reasonably
    > well. I don’t think that’s a serious risk. I’m actually more concerned
    > about someone who deserves acknowledgment not getting it.

I think we should be erring on the side of dealing with that, rather than
worrying too much about gaming.  Nomcom is still randomly selected, afterall.

That's why having multiple paths is a good thing.

I will have to go back and understand what the objections to path (3) were.
I think that in the end, they centered a bit on the number ("6").

---

From: Ted Lemon <[email-masked]>
Date: Sat, 4 Jul 2020 19:32:02 -0400

On Jul 4, 2020, at 7:23 PM, Michael Richardson <[email-masked]> wrote:
> I think that in the end, they centered a bit on the number ("6”).

Quite possibly. I didn’t actually get the sense that there was a consensus against this. There were few very vocal objections, but that’s not the same as a consensus. It felt a bit more like the authors just didn’t want to argue about it, and perhaps felt ill-equipped to evaluate consensus, and so assumed “no” because that would get the job done faster.

---

From: John C Klensin <[email-masked]>
Date: Sun, 05 Jul 2020 01:22:23 -0400

--On Friday, July 3, 2020 17:52 -0400 Ted Lemon
<[email-masked]> wrote:

>> Speaking of "contribution", if something counts as a
>> contribution under BCP 79, isn't the contributor showing
>> active participation in the IETF? Should that make them
>> eligible?
>> 
> I think that BCP 79 contribution is actually what should
> trigger eligibility. It's a slightly higher bar than mere
> attendance. 

I think  there are several traps in the above whether one is
specifically worried about people gaming the system or not.
First, at least as I have understood it, the criteria for nomcom
eligibility were traditionally thought of as approximations or
surrogates for "understands the IETF and how it works".  If is
easy to imagine someone being brought in as an expert on some
particular topic area (estoeric issues in cryptography and/or
internationalization come immediately to mind) and contributing
enough to satisfy not only BCP 79 criteria but even being listed
as a Contributor to an RFC or two without having a clue about
how the IETF, or even a relevant WG, functions.  FWIW, I see a
big different between that sort of Contributor and someone who
actually gets listed as an author or editor of an IETF Stream
RFC: while the contribution may be as great, or actually
greater, getting through AUTH48 with an IETF Stream document
will, in many if not most cases, given such an author a rather
thorough introduction to IETF relationships and processes
sometimes at their most impressive.

We could, of course, rely on the low likelihood of that first
type of contributor not being inclined to volunteer for the
Nomcom even if they were eligible but it seems to me that, if we
start counting likelihood of volunteering or not into
considerations of who should be eligible, we are going to need
to rethink a whole range of criteria and to do so without the
benefit of numbers I think it would be very hard to get and have
plausible.
> 
> The cases I mentioned where people tried to game
> acknowledgments were all cases where the person didn't
> attend in person often enough to qualify that way. But I think
> they were detected and handled reasonably well.

FWIW and coming back to directorate reviews, in the last several
years, I've seen three types of reviews of documents I've been
involved with in one capacity or another:

(1) The reviewer has sufficient subject matter expertise to
contribute substantively to the document in at least some of the
topics it covers.  Those reviews are almost always helpful (even
if they say the document is ok, which would rarely rate being
treated as a BCP 79 Contributor) and almost always rate an
acknowledgement.

(2) The reviewer drew the short straw for a directorate review,
don't not understand the subject matter (and knows it), but does
a careful reading and finds areas of the document when
statements are unclear, typographical errors remain, or other,
typically editorial, nits can be picked.  As an author, editor,
or WG Chair, I appreciate those reviews.  Many of the do not
rise to the level of BCP 79 Contributions and whether they get
explicit acknowledgements is a matter of taste.  Should such a
review be considered as a part of a Nomcom eligibility
qualification?  I don't know: it would indicate some commitment
to the IETF and willingness to do work (good), but not
necessarily any organizational knowledge.

(3) The third type of reviewer doesn't know the subject matter
either but either do not know that, does not believe lack of
knowledge is an impediment, or has really strong opinions and
wants to use the reviewer role as an opportunity to push pet
theories (some of which may be at variance with discussions and
consensus in the relevant WG or more generally).  In any event,
the review, not matter what it says, does not focus
significantly on a careful reading of the document and analysis
of its content.  Occasionally, reviews of this thru out to
identify an important issue that was not considered.  Far more
often, they turn into a huge waste of time as authors, WG
Chairs, document shepherds, and often relevant ADs and/or the
whole working group have to put energy into responding to them.
Contribution in either the BCP 79 sence?  Maybe.  Contribution
in the "value to the IETF" one?  Very rarely if ever.

In my experience at least, virtually everyone who has been very
concerned about whether their efforts get explicit
acknowledgments or not writes reviews that fall into one of the
variations on the third category.

While those distinctions are, I think, interesting and may be
important to the IETF. they make a strong case that
review-writing is not a very good criterion for Nomcom
eligibility if only because separating the categories in some
objective why that the Secretariat could check is likely
impossible.

> I don't
> think that's a serious risk. I'm actually more concerned

I think that is a significant and important issue, but not one
that has (or should have) a lot to do with Nomcom eligibility.

best,
   john

---

From: Ted Lemon <[email-masked]>
Date: Sun, 5 Jul 2020 01:55:43 -0400

It’s hard to see how a person who contributes useful reviews over a period of several years would be less aware of ietf process and culture than a person who shows up for ietf meetings regularly but only listens without contributing. The second person would be qualified under current rules. The first would not. Do you have an argument to offer in favor of this sorry state of affairs?

> On Jul 5, 2020, at 01:22, John C Klensin <[email-masked]> wrote:
> 
> 
>

---

From: S Moonesamy <[email-masked]>
Date: Sun, 05 Jul 2020 05:04:10 -0700

Hi Ted,
At 10:55 PM 04-07-2020, Ted Lemon wrote:
>It's hard to see how a person who contributes useful reviews over a 
>period of several years would be less aware of ietf process and 
>culture than a person who shows up for ietf meetings regularly but 
>only listens without contributing. The second person would be 
>qualified under current rules. The first would not. Do you have an 
>argument to offer in favor of this sorry state of affairs?

Once upon a time, I volunteered to lead a review team.  That team was 
not performing any reviews.  Over time, the team did the work which 
it was expected to do and it became a directorate.  Based on the 
on-list feedback, I would say that the directorate reviews were useful.

I also used to do reviews which did not count as directorate 
reviews.  I was okay if an Area Director viewed that as useless as I 
would consider it as useful input to determine whether it is 
worthwhile for me to perform reviews.

The text for qualification states that: "Volunteers are expected to 
be familiar with the IETF processes and procedures, which are readily 
learned by active participation in a working group and especially by 
serving as a document editor or working group chair."  The word 
"culture" does not appear in the text [1].

Regards,
S. Moonesamy

1. That does not mean that what you wrote is wrong.

---

From: John C Klensin <[email-masked]>
Date: Sun, 05 Jul 2020 13:27:55 -0400

--On Sunday, July 5, 2020 01:55 -0400 Ted Lemon
<[email-masked]> wrote:

> It's hard to see how a person who contributes useful reviews
> over a period of several years would be less aware of ietf
> process and culture than a person who shows up for ietf
> meetings regularly but only listens without contributing. The
> second person would be qualified under current rules. The
> first would not. Do you have an argument to offer in favor of
> this sorry state of affairs?

Absolutely not.  I think "attended meetings" was probably a
useful surrogate for "familiar with the IETF processes and
procedures" and even for "knows many IETF participants and their
work well enough to form sound judgments about whether they
should be in leadership roles" in 1996.   I think that ceased
being true some time ago as both the number of attendees and, I
believe, the fraction of attendees who listened (or sat and did
email without listening) without contributing rose.   I agree
that is a sorry state of affairs.

I also think it is a sorry state of affairs when a Nomcom has to
rely very significantly on questionnaires, voluntary community
comments, and on interviews by less than all of the Nomcom
rather than firsthand knowledge.   Not only is it easy to think
of ways to game that process (especially if there are a few
Nomcom members particularly sympathetic or unsympathetic to a
candidate) but it is just not a substitute for the general
knowledge of randomly-selected members of the community [1].

So, given a choice between having someone who has "contribute[d]
useful reviews over a period of several years" and someone who
manages to registers, pays for, and manages to show up in a
meeting, possibly sits in some rooms where WGs are meeting and
signs blue sheets, but (other than the financial contribution)
is otherwise useless to the IETF, I definitely prefer the
former.  And I prefer such people even if I don't think "does
reviews" is a really good surrogate for "familiar with the IETF
processes and procedures" either.

What I was concerned about when I wrote that note was not
authors of useful reviews but authors of reviews that do not
usefully contribute in any substantive way.  That may
demonstrate no such familiarity and, if someone has decided it
would be good to be on a Nomcom (whether out of a sincere desire
to help the IETF or for less attractive reasons), then writing a
handful of reviews that basically say "I don't see any problems
other than these two typographical errors or nit found by the
nits-checker" is a really easy way to game the "reviewed
documents" criterion, especially when review-writing does not
require any subject matter understanding of the document.   My
concern was just about separately the latter group from the
former.

More generally, however, let's assume there is a person who has
been actively participating in one or more WGs for several
years, reading documents, contributing constructively on mailing
lists and in meetings in ways that lead to improvements and/or
to a better and more diverse informed consensus and who is hence
contributing to the IETF and thereby becoming familiar with how
the IETF works, its culture, etc.  Now assume he or she gets,
f2f, to almost no f2f meetings. If we (continue to) decide that
person should be less qualified to volunteer for the Nomcom than
someone who shows up f2f, listens (or not) to WG discussions,
but does not interact with others in any way that contributes
substantively to the IETF's work, that, too, is a sorry state of
affairs.  Maybe it is even sorry enough that it is time to drop
what draft-carpenter-eligibility-expand-03 calls "Path 1".

   john

[1] Not just randomly-selected members of a pool who are not a
very good sample of the community, and an even less-good sample
of active and contributing participants, even if not being
diliberately gamed or otherwise distorted in any way.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 05 Jul 2020 13:29:13 -0400

> Quite possibly. I didn’t actually get the sense that there was a
    > consensus against this. There were few very vocal objections, but
    > that’s not the same as a consensus. It felt a bit more like the authors
    > just didn’t want to argue about it, and perhaps felt ill-equipped to
    > evaluate consensus, and so assumed “no” because that would get the job
    > done faster.

Brian could say more, or we could read the archives.
I think your analysis is correct.

But, it is interesting what data that we got without it.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 05 Jul 2020 13:48:59 -0400

> sometimes at their most impressive.

Let's assume that this hypothetical person did volunteer, and was selected.
Let me suggest:
  1) they are only 1 of 10 voting members, and it is likely they will
     learn a great deal.  They could become disruptive... but:
     I had to deal with a disruptive voting member as chair, btw, under the
     *current* eligibility rules.

  2) they probably have experience in some other SDO, and maybe they
     will bring some of the good things from that SDO.

  3) if they volunteered, they probably care about the IETF's future.

So, I'm comfortable with your strawman.

---

From: John C Klensin <[email-masked]>
Date: Sun, 05 Jul 2020 16:14:40 -0400

Michael, 

--On Sunday, July 5, 2020 13:48 -0400 Michael Richardson
<[email-masked]> wrote:

>     > surrogates for "understands the IETF and how it works".
> If is     > easy to imagine someone being brought in as an
> expert on some     > particular topic area (estoeric issues in
> cryptography and/or     > internationalization come
> immediately to mind) and contributing     > enough to satisfy
> not only BCP 79 criteria but even being listed     > as a
> Contributor to an RFC or two without having a clue about     >
>     > big different between that sort of Contributor and
> someone who     > actually gets listed as an author or editor
> of an IETF Stream     > RFC: while the contribution may be as
> great, or actually     > greater, getting through AUTH48 with
> an IETF Stream document     > will, in many if not most cases,
> given such an author a rather     > thorough introduction to
> IETF relationships and processes     > sometimes at their most
> impressive.
> 
> Let's assume that this hypothetical person did volunteer, and
> was selected. Let me suggest:
>   1) they are only 1 of 10 voting members, and it is likely
> they will      learn a great deal.  They could become
> disruptive... but:      I had to deal with a disruptive voting
> member as chair, btw, under the      *current* eligibility
> rules.

Please don't misunderstand me.  I, again, think such a person is
unlikely to volunteer and be selected and the odds of several
such people volunteering and have more than one or two be
selected are much smaller.  I tried to say essentially that in
my earlier note.   And I think it is a reasonable hypothesis
that, if such a person decided to volunteer (without some
external pressure or motivation), the reasons would include at
least a willingness to learn.  As with Ted's high-participation
and high-contribution reviewer, I would have assumed, even
before getting your note, that they would bring useful
perspective to the Nomcom.

See below.

>   2) they probably have experience in some other SDO, and
> maybe they      will bring some of the good things from that
> SDO.

I don't see how "probably have experience in some other SDO"
follows from the scenario I outlined.

>   3) if they volunteered, they probably care about the IETF's

Or are curious and interested in learning.  Not quite the same
thing, but see above.
 
> So, I'm comfortable with your strawman.

Given your scenario, so am I.

I'm trying to get at two things here (which I think are
consistent with the general approach of
draft-carpenter-eligibility-expand even if I may disagree about
a detail or two):

(1) Whatever we do should bring us closer to the goal of having
a Nomcom, or at least the vast majority of its voting members,
who are contributing participants in the IETF, familiar with its
policies, procedures, and, with apologies to Subramanian,
culture and, consistent with that, as diverse as possible both
demographically and in terms of different technical and
operational perspectives.  That includes, in particular, getting
away from the assumption that, if two people with equal levels
of contribution, participation, and IETF familiarity might want
to volunteer, it is reasonable to disqualify one of them because
limited travel budgets or because the IETF has chosen
inconvenient (for them) meeting locations or time zones.

(2) However we do that, we need to keep in mind that the nature
of stochastic processes is that sometimes odd things happen.
Therefore, when we say that something would be ok if it is one
person in 10, we'd best have a plan about what to do if is turns
out to be three people or even six people.  That is true whether
that one (or three or six) has wildly different experience (as
in your example), is disruptive or difficult to work with, is
there because they care more about company-provided incentives
or promoting (or blocking) a particular candidate than about the
general health of the IETF.

best,
   john

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 6 Jul 2020 08:48:27 +1200

> I think your analysis is correct.

Well, as far as I know we don't have a chair here since it's not a WG, just
a mailing list, and we're talking about an individual draft unless/until
it gets picked up by a WG or a sponsoring AD. So yes, we dropped path 3
because it seemed to be controversial. (Since I drafted the text, you can
assume I actually think it's a good idea, despite the risk of gaming by
writing valueless reviews.)
 
> But, it is interesting what data that we got without it.

Yes. The data suggest that we can get a decent sized pool of qualified
people with or without counting attendance, and with or without
counting directorate reviews.

    Brian