Subject: Re: [Eligibility-discuss] "Additional Criteria"

From: Ted Hardie <[email-masked]>
Date: Sat, 1 Aug 2020 07:07:27 -0700

On Fri, Jul 31, 2020 at 3:22 PM Stephen Farrell <[email-masked]>
wrote:

>
> I have to say I'd be against that approach. I can't see us
> modifying this that many times, maybe this one experiment
> and once more (perhaps in the schoo WG) in a year or two.
>
> Can I ask a question Ted: For myself, I really do want to
> see us make some moves towards nomcom eligibility for
> people who have been and may continue to be habitually non
> attenders at f2f meetings. Do you share that goal? Or do
> you consider it a non-goal or perhaps even as a thing to
> be avoided?
>
>
I favor nomcom eligibility for any participant who has participated enough
to know the IETF process and community, and I believe that there are
participants who do not habitually attend in person who do know the process
and community.

regards,

Ted

> Cheers,
> S.

---

From: Ted Lemon <[email-masked]>
Date: Sat, 1 Aug 2020 10:44:07 -0400

On Aug 1, 2020, at 10:07 AM, Ted Hardie <[email-masked]> wrote:
> I favor nomcom eligibility for any participant who has participated enough to know the IETF process and community, and I believe that there are participants who do not habitually attend in person who do know the process and community.

I certainly agree with your first point, and I also think that your following observation is accurate. However, you haven’t actually said what conclusion you draw from these assertions. Would you mind saying?

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 1 Aug 2020 17:43:47 +0000

*   I favor nomcom eligibility for any participant who has participated enough to know the IETF process and community, and I believe that there are participants who do not habitually attend in person who do know the process and community.

Let’s go further: *how* familiar do they have to be?  What is the risk?

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 1 Aug 2020 13:59:55 -0400

That (paraphrased as how familiar with the IETF and its processes and 
culture do we want nomcom members to be?) is a good question.  I do not 
know any way to quantify an answer.  Which makes it a difficult 
question.  There is even a fair concern that raising that bar too high 
would bias participation to long term well-connected insiders, avoiding 
which is one of the key goals of the whole nomcom process.

You then ask what is the risk.  There is history of nomcoms that were 
not well-versed in the community and either tried to, or did make 
appointments that turned out to be very bad ideas.  Between the myriad 
factors at work in any specific case, and the appropriate need to 
preserve confidentiality, I do not know any way to be sufficiently 
explicit about the cases I have understood.  But from where I sit it 
looks to be a real problem.

One aspect of the problem is that the channels the nomcom has for 
getting information they lack can be very spotty.  The community is not 
good about giving useful and understandable feedback.  Which is 
understandable as it is both difficult and uncomfortable.  There are 
reports, again hard to substantiate for many reasons, of people refusing 
to give strong negative feedback for fear of it leaking either through 
the nomcom members themselves or through the liaisons and advisors.  And 
while one could modify things so that the advisors do not see the 
feedback, I do not see how the liaisons who are responsible for 
confirming that the process was properly followed can be excluded from 
seeing the feedback.  After all, part of the process is "was the 
feedback considered fairly?"

Another aspect of under-informed nomcoms is related to the discussion we 
have had about liaisons and advisors participating in nomcom 
discussions.  If you have an under-informed nomcom, and if, as seems to 
be the case, the informational channels are lacking, the nomcom is 
likely to turn to and be receptive to input from the liaisons.  However, 
that input is inherently limited, and in some cases has its own agenda 
(for clarity, the vast majority of the liaisons I know about worked very 
hard to be very fair in their information presentations.  You asked 
about risks.)

Yours,
Joel

On 8/1/2020 1:43 PM, Salz, Rich wrote:
>   * I favor nomcom eligibility for any participant who has participated
>     enough to know the IETF process and community, and I believe that
>     there are participants who do not habitually attend in person who do
>     know the process and community.
> 
> Let’s go further: **how** familiar do they have to be?  What is the risk?
> 
>

---

From: Ted Lemon <[email-masked]>
Date: Sat, 1 Aug 2020 14:14:02 -0400

One way to approach this would be to ask, is there a difference in how this risk arises based on the source of the nomcom pool?  Are there ways to mitigate it?

> On Aug 1, 2020, at 14:00, Joel M. Halpern <[email-masked]> wrote:
> 
> That (paraphrased as how familiar with the IETF and its processes and culture do we want nomcom members to be?) is a good question.  I do not know any way to quantify an answer.  Which makes it a difficult question.  There is even a fair concern that raising that bar too high would bias participation to long term well-connected insiders, avoiding which is one of the key goals of the whole nomcom process.
> 
> You then ask what is the risk.  There is history of nomcoms that were not well-versed in the community and either tried to, or did make appointments that turned out to be very bad ideas.  Between the myriad factors at work in any specific case, and the appropriate need to preserve confidentiality, I do not know any way to be sufficiently explicit about the cases I have understood.  But from where I sit it looks to be a real problem.
> 
> One aspect of the problem is that the channels the nomcom has for getting information they lack can be very spotty.  The community is not good about giving useful and understandable feedback.  Which is understandable as it is both difficult and uncomfortable.  There are reports, again hard to substantiate for many reasons, of people refusing to give strong negative feedback for fear of it leaking either through the nomcom members themselves or through the liaisons and advisors.  And while one could modify things so that the advisors do not see the feedback, I do not see how the liaisons who are responsible for confirming that the process was properly followed can be excluded from seeing the feedback.  After all, part of the process is "was the feedback considered fairly?"
> 
> Another aspect of under-informed nomcoms is related to the discussion we have had about liaisons and advisors participating in nomcom discussions.  If you have an under-informed nomcom, and if, as seems to be the case, the informational channels are lacking, the nomcom is likely to turn to and be receptive to input from the liaisons.  However, that input is inherently limited, and in some cases has its own agenda (for clarity, the vast majority of the liaisons I know about worked very hard to be very fair in their information presentations.  You asked about risks.)
> 
> Yours,
> Joel
> 
> -- 
> Eligibility-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/eligibility-discuss

---

From: denis bider <[email-masked]>
Date: Sat, 1 Aug 2020 16:40:27 -0500

General thoughts on community self-leadership:

- How to avoid the Dunning-Kruger effect from causing the election of
someone mediocre (but representative!) is an unsolved problem.

- People are of fundamentally good intent. This can come across wildly
distorted when people are confused, hurt and unskilled.

- A social process does not need to be designed against attack by
super-intelligent aliens. The real challenges are (1) a lack of effective
organization altogether, and (2) sometimes, events caused by the confused,
hurt and unskilled.

- Inflexible and technical rules enable malicious compliance. Malicious
compliance creates a temptation for more rules. A proliferation of rules
has a strangling effect on constructive, well-meant contributions.

- The rules of a social process are interpreted by people, not algorithms.
The rules therefore do not have to be inflexible and technical, and can be
general guides.

- Universal agreement on what the rules mean and how they should be applied
is not required and should not be expected. The Supreme Court hands down
5/4 decisions regularly, that is fine as long as decisions are made and can
be overturned at a later time.

- A social process will not lead to better results than the people who
execute it. It is far more effective to secure the involvement of the best
people, than to exclude offenders by devising rules.

You build the house you want by bringing in the parts that will comprise
the house you want. You don't do it by writing rules to exclude the wrong
parts.

Not only is no one interested in bringing you the wrong parts; but writing
rules to exclude the wrong parts does not actually bring the right ones.

These are probably the only thoughts I will have on this.

Regards,

denis

On Fri, Jul 31, 2020 at 4:20 PM Ted Lemon <[email-masked]> wrote:

> On Jul 31, 2020, at 5:04 PM, Michael StJohns <[email-masked]>
> wrote:
>
> Yes, but for example [email-masked] is an IETF community working
> group mailing list, and [email-masked] is an IETF community mailing list -
> need I go on?
>
>
> Apparently, because I have no idea what your concern is. Do you mean that
> if people participate in RGs, that doesn’t qualify them for nomcom? Well
> then either we include RG mailing lists in the set, or we decide that
> indeed participating in RGs doesn’t qualify you for nomcom. This is just a
> decision to make—there’s nothing inherently difficult about it.
>
> Why is X on the list of qualified people - not why is X on the mailing
> list....    in any event, is "+1" a contribution?   Is "this idea is stupid
> for [list of reasons that have been debunked many many many times]" a
> contribution?   If so, I can automate a random +1 or -1 to be sent to the
> list every say 10 topics.
>
> Yes, you can. If people really want to kill the IETF, they can automate
> bots to kill the IETF. This is possible now—it just isn’t happening because
> nobody is motivated to do it. I agree that this is a potential attack
> surface, but there are things we can do about it: maybe we need to only
> allow addresses registered in the datatracker? Maybe only addresses
> registered in the datatracker that are for people who attended online or in
> person? If we’re so concerned that this is going to happen that we think
> it’s urgent to prevent it, that would make it a lot harder.
>
> But regardless, what you miss with your criteria are actual contributions:
> private notes to the WG chair, or back and forths with the authors on a
> document, or, for the GITHUB based WGs, you might miss pull requests or
> comments in the issues trackers.  Or the document that the contributor was
> working on is awaiting an action by the WG chair, by the AD, or by the
> RPC.  I don't find your criteria either useful or all that related to the
> concept of "contributing”.
>
>
> You know, this is starting to feel like a pissing contest. Yes, what I
> suggested doesn’t capture every contribution, but it does capture an
> important type of contribution. Yes, it can be gamed. And the gaming can be
> mitigated. My concern is that right now we are headed in the direction of
> valorizing “showed up for the meeting” above “participated on the mailing
> list” and I’d like to address that. If you think participating on the
> mailing list doesn’t matter, why are you responding here?
>
>
>
>
> _______________________________________________
> Cfrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/cfrg
>

---

From: Watson Ladd <[email-masked]>
Date: Sat, 1 Aug 2020 17:42:09 -0400

On Sat, Aug 1, 2020, 1:03 AM Richard Outerbridge <[email-masked]> wrote:

> On 2020-07-31 (213), at 17:04:35, Michael StJohns <[email-masked]>
> wrote:
>
> On 7/30/2020 10:34 PM, Ted Lemon wrote:
>
> On Jul 30, 2020, at 9:17 PM, Michael StJohns <[email-masked]>
> wrote:
>
> How do you propose to handle challenges?  E.g.,  "Why am I not on the list
> and what about the three comments I made on that obscure mailing list a
> year ago that obviously mattered?"
>
>
> “That wasn’t an IETF working group mailing list.”
>
>
> So far as I’ve been able to discern, IETF meetings are like the reading
> out of Orders to units on the Board in a game of Diplomacy :)
>

I propose we drop cfrg from this.

Also far fewer backstabbings at the IETF.

__outer

---

From: Michael Richardson <[email-masked]>
Date: Sun, 02 Aug 2020 18:29:51 -0400

Ted Hardie <[email-masked]> wrote:
    > Beginning with IETF 109, for any plenary meeting in which the IETF had no
    > in-person participation, any registrant for the meeting will be treated as
    > an in-person registrant for the purposes of the NomCom rules. The
    > experiment for this will terminate when two in-person meetings have been
    > held after IETF 109 or at the third meeting in 2021, whichever is
    > earlier.

I feel that we already did this for IETF107.
And all the things Ted said.

And you seem to have mis-understood quite a bit of the proposal.
I think that our text was clear enough, but if after this thread you can
provide some clarifying wording, that would be appreciated.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Sun, 02 Aug 2020 18:52:32 -0400

{why is cfrg@ in the CC?}

    > someone mediocre (but representative!) is an unsolved problem.

okay, you are certainly right, but since we are just making a list of
volunteers.  I'm not sure if that mitigates or exacerbates this problem.

    > organization altogether, and (2) sometimes, events caused by the confused,

I think that this is an important point.

There will the threats, but that's why the document does not attempt too many
things at the same time.

    > has a strangling effect on constructive, well-meant contributions.

Agreed.

    > people, than to exclude offenders by devising rules.

Also Agreed.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 02 Aug 2020 19:29:21 -0400

Ted Lemon <[email-masked]> wrote:
    >> Sure, but are they already fed into a database? If not, what tools are needed to do so?

    > We were told for 108 that the blue sheets were being tracked
    > automatically. That presumably means that they are indeed being fed
    > into the database.

I would also assume that is available somewhere in electronic form.

    > If we wanted to do this for in-person meetings, we would need some
    > additional tooling. But there, the fact that you showed up and picked
    > up your badge is a good indication that you were there, so probably we
    > don’t need the blue sheets for in-person meetings.

When you pick up your badge, they scan it.  So the secretariat knows.
We just need to get it all into the DT.

So let me extend this slightly:
  1) what about virtual interim meetings?
  2) what about WGs that never meet "in person"?



1) virtual interim meetings.

These are largely attended by very active people.
Often all draft authors and WG chairs in some way or another.
Given the current makeup, I am not anxious that they might not be counted.

2) WGs that never meet.

CELLAR is one, for which Spencer and I are co-chairs.
Our virtual interim meetings have 7 to 10 people.
Most wind up as authors of documents which are (slowly) making it to RFC.
As such, they would get counted eventually, but it's not every single person.

I will note that these people would not make good nomcom volunteers because
they do not have a good understanding of the culture.  I don't think we need
any rules to either include or exclude them.

---

From: Michael Richardson <[email-masked]>
Date: Sun, 02 Aug 2020 19:32:13 -0400

Alissa Cooper <[email-masked]> wrote:
    > I understand why draft-carpenter-eligibility-expand frames its proposal
    > as a process experiment, but it strikes me that no matter what
    > “experiment” is run we won’t be able to do a rigorous evaluation of the
    > results because we don’t have agreed criteria that we’re optimizing
    > for. That is, we don’t have agreement about what metrics characterize a
    > “good” pool of nomcom volunteers — bigger or smaller, more or less
    > “diverse,” better or worse familiarity with the community, etc. — other
    > than perhaps by comparing to the size of the pool in recent years. With
    > just months to get this done I don’t see a path to agreeing on other
    > metrics. So despite the use of the “experiment” label, I think it will
    > help us conclude this discussion in a timely fashion if we put the idea
    > of experimental evaluation aside.

Social scientists (and cosmologists) deal all the time with being unable to
have control groups, and having to do before-after comparisons.

I know a few people. I'll make some inquiries about getting help.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael Richardson <[email-masked]>
Date: Sun, 02 Aug 2020 19:58:48 -0400

Ted Hardie <[email-masked]> wrote:
    > As I said earlier in the thread, I don't think the risk here is zero, but I
    > do think we could test over a nomcom cycle and recover if we see evidence
    > of this behavior.    There also may be some mitigations we could put in
    > place, but I have not yet thought through that part of the problem space
    > very well.  A lot of my early thinking clearly has some potental to
    > suppress volunteers signing up, which is not desirable.

My initial thinking:
https://mailarchive.ietf.org/arch/msg/eligibility-discuss/Z2eMQi3la9NtKsDQ-WQ-AViyPTM/

My revised thinking:
https://mailarchive.ietf.org/arch/msg/eligibility-discuss/np4WdEICnT3tLe-TwO7tFeoKZaI/

In either view, the majority number of people who were eligible did not volunteer.
This is perhaps the biggest obstacle to getting any information out wrt "see
evidence of this behaviour".

The more I think about this, the more I think that it warrants some
sociological thinking about.

    > Let me suggest, then, that one way to do this is to make serial additions
    > to the eligibility and assess after each one.  Those would not be
    > experiments but would be presumptive permanent additions, but each small
    > enough that we could assess whether it was sufficient or more additions
    > were required.

Assess... how?
I agree that a series of serial additions is a good process.
I wrote about that sometime last month.  I think we should be up-front about
the different things we might add, and then allow the I* to implement each
one or not. (I don't know: IESG?IAB? combined? something else?)

To me, the question is how big is the volunteer pool.
Growing the size of the eligibility pool is not clearly the right metric to me.
Noting how many people are not eligible in 3-of-5, but would be under the
different paths is actually startling.

---

From: Ted Hardie <[email-masked]>
Date: Mon, 3 Aug 2020 09:45:34 -0700

On Sat, Aug 1, 2020 at 7:44 AM Ted Lemon <[email-masked]> wrote:

> I certainly agree with your first point, and I also think that your
> following observation is accurate. However, you haven’t actually said what
> conclusion you draw from these assertions. Would you mind saying?
>

I conclude that there are participants who do not habitually attend in
person who know the process and community, and that they should be eligible
to serve.

regards,

Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Mon, 3 Aug 2020 09:58:10 -0700

Hi Michael,

> provide some clarifying wording, that would be appreciated.

I don't think I'm arguing that the text is unclear, though Stephen and
Brian believe that one point I raised (the meaning of "formal role") is
worth cleaning up in the draft.

I think I'm arguing that the draft is focused on roles that demonstrate
that an individual is known to the community.  Being on the I*, a WG chair,
a BoF chair, a working group document author, etc. generally means that the
current leadership of the community knows who you are and finds your input
valuable.  But that's not what the current system is designed to optimize
for.  It optimizes for inclusion for anyone who knows the community.  While
the set the document designates probably are people who know the community,
there are some corner cases (e.g. the IAB liaisons) and, more importantly,
there are folks who participate in ways that lets them know the community
without necessarily fitting one of the "community knows you" slots.

I want that group to continue to be eligible.  Without that, I fear that
the system will look like a closed loop to folks starting to participate in
the IETF or, worse, it will become one.  Neither is healthy for the IETF in
the long term.

The proposal I made is very simple and because it is based on the original
criteria, it is almost certainly going to include anyone included by either
the in-person meeting criteria or the draft's criteria.  As Alissa pointed
out, there are risks there that weren't part of an in-person meeting
(socket puppet personalities would be a lot harder to pull off if you have
to go to a reg desk to grab a badge).  It may be, as a result, that my
proposal is not the one the group should pursue.  But I believe the goal
is:  we should include anyone who knows the IETF process and community,
even if they have not been selected by the current leadership for any
specific role.

regards,

Ted Hardie

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 3 Aug 2020 19:13:16 +0000

I think Ted’s point is valid, and it seems we have over-emphasized the importance of measurability.  As I wrote earlier, I’d like to see us discuss the risks of “getting it wrong”

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 4 Aug 2020 08:55:07 +1200

Hi Ted,

> But I believe the goal is:  we should include anyone who knows the IETF process and community, even if they have not been selected by the current leadership for any specific role.

To be clear, you think that some form of proof of active participation is *not* required?

Regards
   Brian

On 04-Aug-20 04:58, Ted Hardie wrote:
> Hi Michael,
> 
> On Sun, Aug 2, 2020 at 3:30 PM Michael Richardson <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
> I don't think I'm arguing that the text is unclear, though Stephen and Brian believe that one point I raised (the meaning of "formal role") is worth cleaning up in the draft.
> 
> I think I'm arguing that the draft is focused on roles that demonstrate that an individual is known to the community.  Being on the I*, a WG chair, a BoF chair, a working group document author, etc. generally means that the current leadership of the community knows who you are and finds your input valuable.  But that's not what the current system is designed to optimize for.  It optimizes for inclusion for anyone who knows the community.  While the set the document designates probably are people who know the community, there are some corner cases (e.g. the IAB liaisons) and, more importantly, there are folks who participate in ways that lets them know the community without necessarily fitting one of the "community knows you" slots. 
> 
> I want that group to continue to be eligible.  Without that, I fear that the system will look like a closed loop to folks starting to participate in the IETF or, worse, it will become one.  Neither is healthy for the IETF in the long term. 
> 
> The proposal I made is very simple and because it is based on the original criteria, it is almost certainly going to include anyone included by either the in-person meeting criteria or the draft's criteria.  As Alissa pointed out, there are risks there that weren't part of an in-person meeting (socket puppet personalities would be a lot harder to pull off if you have to go to a reg desk to grab a badge).  It may be, as a result, that my proposal is not the one the group should pursue.  But I believe the goal is:  we should include anyone who knows the IETF process and community, even if they have not been selected by the current leadership for any specific role.
> 
> regards,
> 
> Ted Hardie
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 4 Aug 2020 09:27:08 +1200

Rich,

> I think Ted’s point is valid, and it seems we have over-emphasized the importance of measurability.  As I wrote earlier, I’d like to see us discuss the risks of “getting it wrong”

My overriding concern is not about measurability in terms of metrics, but that the Secretariat *must* be able to check elgibility automatically; in other words, ultimately the criteria must provide a binary choice.

    Brian

---

From: Ted Hardie <[email-masked]>
Date: Mon, 3 Aug 2020 14:33:32 -0700

On Mon, Aug 3, 2020 at 1:55 PM Brian E Carpenter <
[email-masked]> wrote:

> Hi Ted,
>
> > But I believe the goal is:  we should include anyone who knows the IETF
> process and community, even if they have not been selected by the current
> leadership for any specific role.
>
I think  participation is required to get to know the IETF process and
community.  What concerns mean is definitions of "active" that rely on you
having picked up a role that is assigned by the existing powers-that-be.
That has a tendency to reinforce existing networks rather than open things
to new participants, and I worry that this would be the result in this
case.   Even if it only appears to be the case, it could be discouraging.

There are potential gotchas there.  If someone can invent 10 personas and
pony up the remote meeting fees, they can occupy 10 volunteer slots; one of
them might well get through.  You could manage that in a variety of ways
(e.g. by having the confirmation of willingness always be a video
interview), but even with that it's more risky than it was before.  But I
think the openness is more important, especially short-term, while we work
out what the long-term meeting strategy will be (back to the old way,
forward to SHMOO, somewhere in between).

Thanks for putting up with the last minute critique; I appreciate it.

Ted Hardie

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 3 Aug 2020 22:49:35 +0100

Hiya,

> having picked up a role that is assigned by the existing powers-that-be.

I'll note that the relevant powers-that-be for the current
draft encompasses the set of WG chairs via Path 5, so it's
not only the small collection of I* people. But your point
is fair - with draft-03, to be nomcom-eligible, someone has
to give you the nod for something or you have to get to a
few f2f meetings.

> case.   Even if it only appears to be the case, it could be discouraging.

So I think adding 3-out-of-5 f2f or fully online meetings
could solve that problem for the purposes of this specific
experiment. (I.e. modify path1 to say that.)

> think the openness is more important, 

That's where I think the current draft has an advantage.
But, would it help to specifically allow for a challenge
to selection for nomcom on the basis that the objector
alleges that the person selected has no actual history
of participation in the real work of the IETF, with the
nomcom-chair to investigate and report back?

Could be quite messy, but maybe it could be done.

Cheers,
S.

> especially short-term, while we work

---

From: Ted Hardie <[email-masked]>
Date: Mon, 3 Aug 2020 15:26:45 -0700

On Mon, Aug 3, 2020 at 2:49 PM Stephen Farrell <[email-masked]>
wrote:

> If I understand you correctly, that would eliminate 107, but include 108
and any later all-online meeting.  That would work for me, but also see
below.

> If you rely on challenges, the risk remains pretty high because the same
individual can join 10 different mailing lists under the different names
and ask clarifying questions or similar to provide evidence of their
reality.  Unless somebody notices that the 10 different people share some
other characteristic, identifying them all as the same won't work as each
element of participation is real.  That's why I mentioned the video
interview--you'd have to commit to having whoever did the video interview
be on the NomCom calls, which moves from "sock puppet" to "paid rep"
territory (still an issue, but it drives the cost up for the attacker,
which is good).  It's tedious for the chair to confirm them all, of course,
though you might theoretically ask for an upload of a video confirming
eligibility along with a per-volunteer generated phrase.

But we are heavily into engineering the process here, where I am not sure
there is general agreement on the principle.  If there is, I think we can
square something away that's good enough for a test period.

regards,

Ted Hardie

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 4 Aug 2020 13:41:44 +1200

Ted,

> where I am not sure there is general agreement on the principle. 

No, and since this is only a discussion list with no designated chairs, we don't have anyone whose job it is to extract consensus.

Alissa, or any other handy AD: could we have some input on how to go forward on this?

Regards
   Brian

On 04-Aug-20 10:26, Ted Hardie wrote:
> On Mon, Aug 3, 2020 at 2:49 PM Stephen Farrell <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
> If I understand you correctly, that would eliminate 107, but include 108 and any later all-online meeting.  That would work for me, but also see below.
>  
> 
>     >
> If you rely on challenges, the risk remains pretty high because the same individual can join 10 different mailing lists under the different names and ask clarifying questions or similar to provide evidence of their reality.  Unless somebody notices that the 10 different people share some other characteristic, identifying them all as the same won't work as each element of participation is real.  That's why I mentioned the video interview--you'd have to commit to having whoever did the video interview be on the NomCom calls, which moves from "sock puppet" to "paid rep" territory (still an issue, but it drives the cost up for the attacker, which is good).  It's tedious for the chair to confirm them all, of course, though you might theoretically ask for an upload of a video confirming eligibility along with a per-volunteer generated phrase. 
> 
> But we are heavily into engineering the process here, where I am not sure there is general agreement on the principle.  If there is, I think we can square something away that's good enough for a test period.
> 
> regards,
> 
> Ted Hardie
> 
>  
>

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 02:39:46 +0000

>    My overriding concern is not about measurability in terms of metrics, but that the Secretariat *must* be able to check elgibility automatically; in other words, ultimately the criteria must prov…

What would happen if we let anyone into the nomcom pool who wants it.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 3 Aug 2020 22:43:23 -0400

> What would happen if we let anyone into the nomcom pool who wants it.

We don’t know.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:00:47 +0000

>    We don’t know. 

I work in security.  Let's try to estimate what is the worst that could happen, and see how much effort we need to expend to prevent that from happening.

On the plus side, we'd be more inclusive and get fresh voices.  On the minus side, we might have some folks drop out. We might get some bad choices for AD, etc., but we've weathered those before, and it's not clear that we'd get more bad AD's, etc., standing for the position than we usually do.

Perhaps we should have a tutorial "so you want to be on nomcom or in the IETF leadership"

---

From: Ted Lemon <[email-masked]>
Date: Tue, 4 Aug 2020 12:05:48 -0400

> I work in security.  Let's try to estimate what is the worst that could happen, and see how much effort we need to expend to prevent that from happening.

Isn’t that what we’ve been doing? But I agree that we should do it more explicitly.

> On the plus side, we'd be more inclusive and get fresh voices.  On the minus side, we might have some folks drop out. We might get some bad choices for AD, etc., but we've weathered those before, a…

Well, if we got a really bad nomcom, we could get a _lot_ of I* folks we’d wish we hadn’t gotten, not just a few. I guess the sitting IAB could exercise veto power.

> Perhaps we should have a tutorial "so you want to be on nomcom or in the IETF leadership”

That would certainly be worthwhile. I could have used one. Of course, few are qualified to write it.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 4 Aug 2020 12:20:38 -0400

As a security person (you brought it up), I would expect more 
imagination for "bad results".
As a simple first example, some organization that is not a corporate 
affiliation (so not prevented by other rules) could encourage all their 
members to volunteer for nomcom to make a change in the way the IETF 
works to better meet their needs.  Whether this was a bank industry 
association, a government association, some competing SDO, or some open 
soruce alliance doesn't matter.  It could easily produce results that 
are very bad for the IETF.

Unless you want to argue that the entire nomcom process is largely 
irrelevant, I think your suggestion of wide open allowance clearly does 
have the potential for very bad results.

Yours,
Joel

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:23:22 +0000

*   Isn’t that what we’ve been doing? But I agree that we should do it more explicitly.

In my view, that’s not what we’re doing.  We’re discussing eligibility criteria. I am suggesting there be none.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:31:42 +0000

I don't see how removing the attendance (or similar) requirement changes this.  Can you explain?  What stops the ABA from having their members flood the pool now?  The only difference currently is "it takes two years," and work here seems to be aimed at reducing that time investment.

---

From: Ted Lemon <[email-masked]>
Date: Tue, 4 Aug 2020 12:34:46 -0400

> I don't see how removing the attendance (or similar) requirement changes this.  Can you explain?  What stops the ABA from having their members flood the pool now?  The only difference currently is…

We’ve talked about e.g. requiring both that nomcom members participate on mailing lists over a long period of time, which is fake-able but expensive to fake, and also that they attend some number of in-person or online full IETF meetings, which is also fake-able but expensive to fake. The time investment has the additional benefit that if some organization decided to try to do this, we would see it in the data long before they succeeded, and could consider prophylactic measures.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:39:06 +0000

*   We’ve talked about e.g. requiring both that nomcom members participate on mailing lists over a long period of time, which is fake-able but expensive to fake

Bots, going back to Serdar Argic on Usenet, are not expensive.

  *   The time investment has the additional benefit that if some organization decided to try to do this, we would see it in the data long before they succeeded, and could consider prophylactic measures.

IETF participation as a thought-crime?  I’m sure that’s not what you mean.

---

From: Ted Lemon <[email-masked]>
Date: Tue, 4 Aug 2020 12:43:56 -0400

On Aug 4, 2020, at 12:39 PM, Salz, Rich <[email-masked]> wrote:
> We’ve talked about e.g. requiring both that nomcom members participate on mailing lists over a long period of time, which is fake-able but expensive to fake

And are an attack, which can be recognized and countered, given sufficient time.
 
> The time investment has the additional benefit that if some organization decided to try to do this, we would see it in the data long before they succeeded, and could consider prophylactic measures. 

It’s not. If we start to notice trends in future nomcom eligibility data, we might start to look harder at whether people are participating or sockpuppeting. But let’s be honest: if the ABA decided to take over nomcom, they could already do it with in-person meeting rules. It would just cost more. The ABA is probably not the right threat model.

Also, if the IETF really got pwned, the people doing useful work would walk, and the IETF would lose its legitimacy. Any attack on the IETF has to take that into account in order to be successful. A symbiote is just a parasite that’s useful.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:46:38 +0000

*   The ABA is probably not the right threat model.
ABA could mean American Banking Association or American Bar Association; I was deliberately ambiguous. :)

  *   Also, if the IETF really got pwned, the people doing useful work would walk, and the IETF would lose its legitimacy. Any attack on the IETF has to take that into account in order to be successful. A symbiote is just a parasite that’s useful.

It needn’t be “all or nothing” takeover. It might not even be noticeable.

But again, curious why we need any attendance requirement and what the risk is if we dropped it.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 4 Aug 2020 12:52:40 -0400

Are you asking why we "need" attendance requirements, as you say in this 
message, or why we need any requirements, as you said in an earlier 
message.  The two are not interchangeable.

Yours,
Joel

On 8/4/2020 12:46 PM, Salz, Rich wrote:
>   * Also, if the IETF really got pwned, the people doing useful work
>     would walk, and the IETF would lose its legitimacy. Any attack on
>     the IETF has to take that into account in order to be successful. A
>     symbiote is just a parasite that’s useful.
>

---

From: Ted Lemon <[email-masked]>
Date: Tue, 4 Aug 2020 12:53:20 -0400

> But again, curious why we need any attendance requirement and what the risk is if we dropped it.

Attendance requirements connect online identification to actual identification. Makes it harder to sock-puppet, and makes the things you’d do to sock-puppet crimes.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 16:54:12 +0000

Thanks for catching me on that.  I was wrote hastily.  (In an IETF message, imagine that.)

I think both questions are worth thinking about.  I think I am in favor of no requirements to join the pool.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 4 Aug 2020 18:54:36 +0000

*   Attendance requirements connect online identification to actual identification. Makes it harder to sock-puppet, and makes the things you’d do to sock-puppet crimes.

No crime needed, just willingness to invest in membership fees.  Yes, you’d have to show up. Seems to me that most people just state their name and don’t show ID.  And, since the current set of proposals are adding online participation as qualification options, it’s even easier. And certainly no barrier to a national-scale adversary or one able to firewall an entire country.

I didn’t use the term sock puppet, I said a bot. And I was talking solely in terms of how easy it is to do online participation.

But again, I think the key point is not just the ease of surmounting obstacles, but rather some thinking about what if there were no obstacles. If only because at that point we can think of what the harms are, and effective means of preventing them.  If the nomcom pool increases by 50% because anyone can put their name in, then what happens if one-third of the voting members are newcomers?

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 5 Aug 2020 11:48:47 +1200

> are very bad for the IETF.

My preferred example is: what would have happened if the NomCom window
had been open on that basis during the fuss about the Ethos Capital bid
for .org? Don't imagine that the thousands of people who signed that
petition would have been aware of the difference between ICANN, ISOC,
PIR and the IETF, because they certainly weren't. We'd have been totally
swamped by random volunteers.

Quite apart from that (concerted enemy action) there is another risk that
is in fact close to a certainty: a substantial proportion of NomCom
would be people with little knowledge or understanding of the IETF process,
the IETF social structure, and of the past record of any of the candidates.
In other words, a pretty useless NomCom.

So I am about 180 degrees away from Rich on this proposal. I couldn't
disagree more.

    Brian

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 4 Aug 2020 16:57:02 -0700

> So I am about 180 degrees away from Rich on this proposal. I couldn't

I have to agree with Brian here. I think the objective embodied in the
current criteria -- namely that people should have some investment in and
understanding of the IETF before they serve on the nomcom -- is a worthy
one. It appears that the criteria themselves may no longer be properly
achieving that (though it seems like a different question whether they
would in a post-COVID world), so we should consider revising them. However,
I think whatever revision we do should still try to aim for that same
objective.

-Ekr

---

From: Michael StJohns <[email-masked]>
Date: Tue, 4 Aug 2020 20:14:07 -0400

> So I am about 180 degrees away from Rich on this proposal. I couldn't

It's possible I'm missing something here, but AFAICT, you would have to 
have the people in your example be active at at least 3 different times, 
so yes, one virtual IETF meeting might end up with 1000 or so random 
"fake" attendees, but would 3 of 5 end up with the same 1000 fake 
attendees over a period of a year?

I'm in agreement that we want people with some knowledge of the IETF, 
but we no longer have the luxury of having a rough equivalence of 3 of 5 
in person meetings equals "some knowledge of the IETF".   I haven't been 
too impressed with the various alternatives proposed because they either 
include subjective measures or include objective measures that might be 
difficult to track or evaluate.    The costs with those approaches are 
rapidly exceeding the possible benefits.  And to be even more frank, I 
think we've already had at least a few Nomcom's where the majority of 
the voting members had much less knowledge of the IETF than we'd prefer.

For the next year (meetings108-110), let's count everyone who registers 
for the IETF and shows up in at least one session towards 3 of 5.  
Shortly after meeting 110 and before the selection of the Nomcom, let's 
do a review of how that eligible pool compares with the most recent 
eligible pool with an eye to identifying potential gaming and sock 
puppet scenarios - we may have better ideas about eligibility criteria 
at that point.

Perhaps we also add a way for people to register themselves (e.g. you 
must provide a video assertion of your identity when you volunteer for 
the Nomcom).  "Hi, I'm Mike StJohns.  My email address is 
[email-masked], my organization for Nomcom purposes is 
Independent and I've attended at least N meetings". Those can then help 
us deal with the trivial cases of one person multiple identities.

Or we can throw out the current Nomcom model, and do something like 1/2 
the members have to have 11 of 20 meetings while the others need 3 of 
5.  Or we fix the model that's had more than 1/2 the Nomcom voting 
members come from just 4-6 companies over at least the last 8 years.

Later, Mike

---

From: Ted Lemon <[email-masked]>
Date: Tue, 4 Aug 2020 20:17:27 -0400

On Aug 4, 2020, at 8:14 PM, Michael StJohns <[email-masked]> wrote:
> It's possible I'm missing something here, but AFAICT, you would have to have the people in your example be active at at least 3 different times, so yes, one virtual IETF meeting might end up with 1000 or so random "fake" attendees, but would 3 of 5 end up with the same 1000 fake attendees over a period of a year?

Rich is suggesting that we accept all volunteers.

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 4 Aug 2020 21:48:10 -0400

Well, as I said, I fear we will run out of time if we attempt to get agreement on principle in advance of the selection of the 2021 nomcom. I also note the very small number of people who have engaged in this discussion thus far. I’d really like to get to the point of having a draft that is ready for broader review.

In reviewing the recent emails on the list, I wonder if there is a compromise that could be satisfactory that combines Ted’s proposal with the other criteria listed in the draft and limits the experiment to one year to minimize the threat of gaming. The resulting paths to eligibility would look something like the following (using the draft’s numbering):

Path 1: The person has attended 3 out of the following 5 IETF meetings: 106, 107, 108, 109, 110. For meetings held entirely online, online attendance counts as attendance.

Path 2: Has been a WG Chair or Secretary within the last 3 years.

Path 4: Has served in the IESG or IAB, or has been appointed to a formal role by the IESG or IAB, within the last 5 years.

Path 5: Has been a listed author of at least 2 IETF stream RFCs within the last 5 years.  A draft that has been approved by the IESG and is in the RFC Editor queue counts.

Notes about this proposal:

- The 2021 nomcom will be seated between IETF 110 and IETF 111.

- Path 1 has more breadth than the draft proposes, so hopefully that would be welcome by those who have been advocating for a larger pool, while also continuing to emphasize the recency of engagement which the current criteria values.

- With our current tools we have the ability to determine whether people attended in-person and online meetings (not just whether they registered).

- Because we are deciding on these criteria after IETF 108 and because Path 1 requires attendance at 3 out of 5 meetings, Path 1 does not create real avenues for gaming that I can see. People who wanted to game the system would have needed to start already with IETF 108 or 107, and it seems unlikely for that to have happened.

- The proposals around adding mailing list participants, document reviewers, and document contributors seem sufficiently controversial that I’m skeptical about adding them to a compromise proposal.

If people are interested in this, we could re-run the numbers and diagrams by somehow incorporating 107 and 108 (e.g., selecting people who attended three out of the last five meetings as of today, or two out of the last three) to see what they produce.

Alissa

> On Aug 3, 2020, at 9:41 PM, Brian E Carpenter <[email-masked]> wrote:
> 
> Ted,
>

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 05 Aug 2020 14:39:34 +1000

> Path 5: Has been a listed author of at least 2 IETF stream RFCs within the last 5 years.  A draft that has been approved by the IESG and is in the RFC Editor queue counts.

I support this model.  It's not perfect but it expands the pool and is practical.

I think it's worth trying to get consensus on the platonic ideal model that we want, which I believe can be minimally summed up as:

*1) is a distinct human being (one person, one "vote" - aka ticket in the lottery)**
*
*2) has an understanding of the IETF*

I think everybody agrees on number 1 - we to confirm that each eligible entrant into the nomcom selection process corresponds to exactly one human being.

I'm not seeing consensus on how deep the understanding of the IETF needs to be.  There's a stronger form of this which is "has the IETF's interests at heart" - which is difficult going on impossible to prove, but is definitely something we all want.

I believe that it's important to validate the "one human" before the lottery, not afterwards, as it's much cheaper to enter many heisenburg-humans into the pool and resolve whichever one gets picked into a human being than to collect multiple actual humans.

Beyond that, I'm pretty happy to keep expanding the definition of "has an understanding of the IETF" out widely and see what happens.  We have a process for recalls and if it became clear that more than 2 people had the same affiliation and concealed it that would be grounds to overturn their decisions. 

I'd be in favour of strengthening that restriction to the point that if more than 2 people of the same affiliation were selected and they fail to disclose their relationship then they all lose their authority.  (not for the case like the recent nomcom issue where somebody disclosed immediately on realising the problem, but if they make a set of decisions as a bloc and then can be shown to have done so).

In summary - I believe I agree with Rich here, but with some stronger protection around "identify a human" than just "accept every unique email address which shows up".

Bron.

--
  Bron Gondwana, CEO, Fastmail Pty Ltd
  [email-masked]

---

From: Ted Lemon <[email-masked]>
Date: Wed, 5 Aug 2020 08:25:48 -0400

> I'd be in favour of strengthening that restriction to the point that if more than 2 people of the same affiliation were selected and they fail to disclose their relationship then they all lose thei…

I generally agree with your analysis, Bron. One thing I would point out about this aspect of it, though, is that getting rid of nomcom members is /expensive/. We have a couple of spares, but if we had to take down four nomcom members, we’d have to bring in replacements, and a great deal of work would be lost. So it’s pretty important to surface these issues before nomcom is chosen, not after.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 05 Aug 2020 23:02:28 +1000

> I generally agree with your analysis, Bron. One thing I would point out about this aspect of it, though, is that getting rid of nomcom members is /expensive/. We have a couple of spares, but if we…

Yes, that's pretty expensive - but it would only happen once and we would close the hole.  Basically all I think we need is an emergency brake which can be pulled if some entity manages to stack half the nomcom and fill the entire I* with their toadies, which should hopefully be pretty obvious.

I don't think we need to make the cost of trying this kind of attack too high so long as we make it possible to stop it when it's detected.  There's still a few motivated individual actors who I could imagine trying to drop their own number of rubber stamps into the I* but the parties who can afford even a moderate attack are also parties who have too much reputation risk unless they're guaranteed a win.

Bron.

---

From: Ted Lemon <[email-masked]>
Date: Wed, 5 Aug 2020 09:23:35 -0400

> Yes, that's pretty expensive - but it would only happen once and we would close the hole.  Basically all I think we need is an emergency brake which can be pulled if some entity manages to stack ha…

If we would be able to detect it after it happened once and close the hole, why not close the hole first? What would be different /after/ it happened than before?

I mean, if you really want to close this hole, you would want nomcom participants to state their affiliation in an affadavit, and declare their participation in other professional orgs and SDOs as well. But what would be the grounds for a lawsuit when they lied? Could they be charged with a crime? Is this really practical?

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 5 Aug 2020 13:34:21 +0000

>    Rich is suggesting that we accept all volunteers.

Well, sort of.  I'm asking that we talk about it.  So far it doesn't have much support.

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 5 Aug 2020 13:59:08 +0000

*   I generally agree with your analysis, Bron. One thing I would point out about this aspect of it, though, is that getting rid of nomcom members is /expensive/

The easy fix is to list a dozen alternates, right?

---

From: Ted Lemon <[email-masked]>
Date: Wed, 5 Aug 2020 10:01:44 -0400

> The easy fix is to list a dozen alternates, right?

The alternates have to participate, but don’t get to vote.  This is a rather thankless task.

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 5 Aug 2020 14:08:20 +0000

*   The easy fix is to list a dozen alternates, right?

  *   The alternates have to participate, but don’t get to vote.  This is a rather thankless task.

Looking at RFC 8713 there’s only one instance of “alternate” and it doesn’t mean this.  What am I missing?

Unfortunately, 8713 also talks about how to replace nomcom members and it says re-run the randomization.  I think we should not do that, but have a list of “pinch hitters” available.

---

From: Ted Hardie <[email-masked]>
Date: Wed, 5 Aug 2020 07:56:32 -0700

Howdy,

A few comments below, but the summary is that this compromise works for me.

On Tue, Aug 4, 2020 at 6:48 PM Alissa Cooper <[email-masked]> wrote:

> Well, as I said, I fear we will run out of time if we attempt to get
> agreement on principle in advance of the selection of the 2021 nomcom. I
> also note the very small number of people who have engaged in this
> discussion thus far. I’d really like to get to the point of having a draft
> that is ready for broader review.
>
> In reviewing the recent emails on the list, I wonder if there is a
> compromise that could be satisfactory that combines Ted’s proposal with the
> other criteria listed in the draft and limits the experiment to one year to
> minimize the threat of gaming. The resulting paths to eligibility would
> look something like the following (using the draft’s numbering):
>
> Path 1: The person has attended 3 out of the following 5 IETF meetings:
> 106, 107, 108, 109, 110. For meetings held entirely online, online
> attendance counts as attendance.
>
> Path 4: Has served in the IESG or IAB, or has been appointed to a formal
> role by the IESG or IAB, within the last 5 years.
>
> I still believe that it would be useful to define "formal" here.  It may
also be important to distinguish between whether the appointment occurred
within the last five years or the service occurred within the last five
years.  For some IANA experts, we have had some appointees serve for many
years, and it is not clear whether they would be included.  In order to
make the checks easier, I would personally suggest sticking to confirming
that the appointment was made within the last 5 years, but it needs to be
clear either way.

> Path 5: Has been a listed author of at least 2 IETF stream RFCs within the
> last 5 years.  A draft that has been approved by the IESG and is in the RFC
> Editor queue counts.
>
> Given the other appointments made by the NomCom, I would suggest formally
including IAB documents here, though a quick check shows that this would
not actually change the pool because the non-IAB authors over the past 5
years would either qualify under the other rules or would not have 2
documents.

Notes about this proposal:
> - Path 1 has more breadth than the draft proposes, so hopefully that would
> be welcome by those who have been advocating for a larger pool, while also
> continuing to emphasize the recency of engagement which the current
> criteria values.
>
> - With our current tools we have the ability to determine whether people
> attended in-person and online meetings (not just whether they registered).
>
> - Because we are deciding on these criteria after IETF 108 and because
> Path 1 requires attendance at 3 out of 5 meetings, Path 1 does not create
> real avenues for gaming that I can see. People who wanted to game the
> system would have needed to start already with IETF 108 or 107, and it
> seems unlikely for that to have happened.
>
> - The proposals around adding mailing list participants, document
> reviewers, and document contributors seem sufficiently controversial that
> I’m skeptical about adding them to a compromise proposal.
>
> If people are interested in this, we could re-run the numbers and diagrams
> by somehow incorporating 107 and 108 (e.g., selecting people who attended
> three out of the last five meetings as of today, or two out of the last
> three) to see what they produce.

While I would be interested to see the numbers and diagrams, I support this
now.

regards,

Ted Hardie

> > No, and since this is only a discussion list with no designated chairs,
> we don't have anyone whose job it is to extract consensus.
> >
> >> On Mon, Aug 3, 2020 at 2:49 PM Stephen Farrell <
> [email-masked] <mailto:[email-masked]>> wrote:
> >>
> >>
> >> If I understand you correctly, that would eliminate 107, but include
> 108 and any later all-online meeting.  That would work for me, but also see
> below.
> >>
> >>
> >>>
> >> If you rely on challenges, the risk remains pretty high because the
> same individual can join 10 different mailing lists under the different
> names and ask clarifying questions or similar to provide evidence of their
> >> But we are heavily into engineering the process here, where I am not
> sure there is general agreement on the principle.  If there is, I think we
> can square something away that's good enough for a test period.
> >>
> >> regards,
> >>
> >> Ted Hardie
> >>
> >>
> >>

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 5 Aug 2020 11:27:52 -0400

> On Aug 5, 2020, at 10:56 AM, Ted Hardie <[email-masked]> wrote:
> 
> Howdy,
> 
> On Tue, Aug 4, 2020 at 6:48 PM Alissa Cooper <[email-masked] <mailto:[email-masked]>> wrote:
> I still believe that it would be useful to define "formal" here.  It may also be important to distinguish between whether the appointment occurred within the last five years or the service occurred within the last five years.  For some IANA experts, we have had some appointees serve for many years, and it is not clear whether they would be included.  In order to make the checks easier, I would personally suggest sticking to confirming that the appointment was made within the last 5 years, but it needs to be clear either way.

All good points, and I wonder whether it might be simpler to drop the “has been appointed to a formal role” bit. Collecting this list will require a lot of manual work because there are a bunch of different appointment processes and the roles are not maintained in the datatracker. It’s also duplicative, because many of the people appointed to formal roles are also working group chairs or document authors or both.

Alissa

> Given the other appointments made by the NomCom, I would suggest formally including IAB documents here, though a quick check shows that this would not actually change the pool because the non-IAB authors over the past 5 years would either qualify under the other rules or would not have 2 documents.
> 
> While I would be interested to see the numbers and diagrams, I support this now. 
> 
> regards,
> 
> Ted Hardie
> 
> 
>  
> Alissa
> 
> > On Aug 3, 2020, at 9:41 PM, Brian E Carpenter <[email-masked] <mailto:[email-masked]>> wrote:
> > 
> > Ted,
> > 
> >> On Mon, Aug 3, 2020 at 2:49 PM Stephen Farrell <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
> >> 
> >> 
> > [email-masked] <mailto:[email-masked]>
> > https://www.ietf.org/mailman/listinfo/eligibility-discuss <https://www.ietf.org/mailman/listinfo/eligibility-discuss>
>

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 6 Aug 2020 08:48:09 +1200

On 06-Aug-20 03:27, Alissa Cooper wrote:
> 
>> On Aug 5, 2020, at 10:56 AM, Ted Hardie <[email-masked] <mailto:[email-masked]>> wrote:
>>
>> Howdy,
>>

Also, it's easier to extend eligibility later than to reduce it, so if we drop this for now the decision is reversible. Perhaps adding these roles to the datatracker could be on a wish list.

   Brian

---

From: Michael StJohns <[email-masked]>
Date: Wed, 5 Aug 2020 17:45:29 -0400

> Well, sort of.  I'm asking that we talk about it.  So far it doesn't have much support.
I could have sworn you were implying .... ah well, I got lost in the thread.

In any event, it does have the virtue of simplicity!  But I agree with 
Brian that it's got way too many downsides and no mitigations.

Later, Mike

---

From: "Salz, Rich" <[email-masked]>
Date: Wed, 5 Aug 2020 21:49:00 +0000

Brian listed "not familiar with the IETF" as a downside.  Were the others that I missed?

As for no mitigations, the challenge procedure (sec 4.17 of RFC 8713) is not sufficient?

---

From: Michael StJohns <[email-masked]>
Date: Wed, 5 Aug 2020 18:06:40 -0400

> As for no mitigations, the challenge procedure (sec 4.17 of RFC 8713) is not sufficient?
Brian's example (and I'm paraphrasing here rather than cutting and 
pasting) of 1000 people who confused us with the ISOC  (of which 900 are 
people who don't work for companies that normally show up at the IETF) 
and decided to volunteer for the Nomcom because they were annoyed with 
the sale of the .ORG domain and wanted to have it be reversed (not 
understanding that that's not what we do).  I don't know how likely it 
is, but it's possible.

I'm not sure how the challenge  procedure would apply as that only 
appears to apply to mis-stated affiliations or - as was the most recent 
case - a selection process not consistent with the rules?

Mike

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 5 Aug 2020 18:08:36 -0400

If there are no criteria, then there is no basis for challenge.  So no, 
the challenge procedure does not help any of the concerns that have been 
expressed with having no criteria.

And yes, multiple concerns were expressed.  Whether you agree with them 
does not change that they were expressed.  And does not create an 
obligation for us to recapitulate them.

Yours,
Joel

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 06 Aug 2020 08:22:48 +1000

> If we would be able to detect it after it happened once and close the hole, why not close the hole first? What would be different /after/ it happened than before?

Because the hole is somewhat theoretical and the restrictions also exclude real people.  We do need to have a backstop against the hole, but not at the expense of excluding a wide swath of the community.

> I mean, if you really want to close this hole, you would want nomcom participants to state their affiliation in an affadavit, and declare their participation in other professional orgs and SDOs as…

The grounds aren't so much for a lawsuit against them for lying (a problem we already have for both "is a real human" and "no more than 2 from same affiliation") - it's a defense against a lawsuit from them for improper removal if/when they get removed.

Bron.

---

From: Ted Lemon <[email-masked]>
Date: Wed, 5 Aug 2020 19:10:26 -0400

> Because the hole is somewhat theoretical and the restrictions also exclude real people.  We do need to have a backstop against the hole, but not at the expense of excluding a wide swath of the comm…

It’s really hard to imagine a situation where a bunch of people you want to include could be excluded by a mitigation to an attack. Can you come up with an example?

> The grounds aren't so much for a lawsuit against them for lying (a problem we already have for both "is a real human" and "no more than 2 from same affiliation") - it's a defense against a lawsuit…

If there’s no risk associated with trying, then why not try? I find it unlikely that anyone would sue us for being excluded, but if they were to do so, the fact that the criteria changed after selection would mean that this probably wouldn’t protect us. Of course, IANAL.

---

From: John C Klensin <[email-masked]>
Date: Thu, 06 Aug 2020 00:03:54 -0400

Hi.  I just read most of this thread-- an activity which given
the relatively small number of participants who seem to be
repeating themselves (possibly because of a possibly accurate
perception that some others are too busy talking to listen and
think) -- was fairly painful.  Based on that, some observations
(several borrowed from people with whom I'm rumored to always
disagree).

(1) If our objective/ ideal is to have a Nomcom volunteer pool,
and hence a Nomcom, made up of people with a thorough
understanding of the IETF and how it works, of the current
leadership, and of most, if not all, of the people who are
potential members of the leadership, the ability to show up at a
meeting and pick up a bag has, over time, become a fairly rotten
surrogate for that.   That is actually fairly easy to
demonstrate if not to quantify by looking at the reliance on
ever longer and more complex candidate questionnaires that
mostly ask for information that would have been first-hand
Nomcom knowledge a quarter-century ago.    The good news is that
doing better on that "thorough understanding" criterion is not
hard; we certainly should strive to not do worse.

(2) To me, notions based on "we do something special for
meetings that are entirely online and then go back to normal for
f2f meetings" (including the current Path 1 proposal as I
understand it) are very high risk.  It is just a guess, but I
think that, independent of the virus, quarantine, and travel ban
issues, we are going to see enough economic and organizational
fallout -- even if there is a good vaccine-- that it is almost
inevitable that we will see national and corporate/organization
travel restrictions and increased requirements for travel
justification for some time, probably years.  If I'm right, that
will leave the LLC with a choice between saying that we don't do
f2f meetings until the "normal" profile of people can be present
(again, potentially a few years) or holding them for whomever
can attend.  For the latter case, we'd better have a plan to
count remote participants --and making sure that Nomcoms don't
have even an implicit f2f meeting requirement for voting member
participation -- even if meetings are f2f starting with IETF 109
or we are likely to find ourselves looking at very homogeneous
Nomcoms.

(3) I'm also a little concerned about participants, even active
participants, who are only involved in a single WG.    People
who come to the IETF to participate in a narrowly-focused
activity, especially one dominated by small number of companies
with common interests, may have even less understanding of the
IETF than the average of people who get someone else to pay
travel and registration fees and then show up and collect
badges.  See comment above about not making things worse.

(4) all we can control directly is eligibility to volunteer to
be part of the pool from which the Nomcom is drawn.  But the
people from that pool who actually volunteer are not a
representative of the pool.  Maybe my concern in (3) above is a
non-issue because, if someone only cares about one WG (or has
support only for participation in that WG), they will not
volunteer for the Nomcom whether made eligible or not... at
least absent company efforts to pack the Nomcom as a means of
packing the I*.

best,
   john

---

From: "Bron Gondwana" <[email-masked]>
Date: Thu, 06 Aug 2020 14:55:17 +1000

> least absent company efforts to pack the Nomcom as a means of

How do people feel about making the volunteering a little more "opt-out"?

Being involved in this body has significant benefits to those doing it, and while you do have to pay money to show up to face-to-face meetings, it has otherwise been "free" to contribute and get the benefits of access to and review by many people for your ideas - even more so if they get adopted by a working group and published.  It seems fair to expect you to take on a risk of "jury duty" as part of your "citizenship".

Because I agree with John - right now we only have a distinctly smaller group of people volunteering than those contributing, and that does skew the demographic that the random number get to pick from.  We should strongly discourage not being in that pool.

Bron.

---

From: John C Klensin <[email-masked]>
Date: Thu, 06 Aug 2020 02:12:40 -0400

--On Thursday, August 6, 2020 14:55 +1000 Bron Gondwana
<[email-masked]> wrote:

>> representative of the pool.  Maybe my concern in (3) above is
>> a non-issue because, if someone only cares about one WG (or
>> has support only for participation in that WG), they will not
> Being involved in this body has significant benefits to those
> doing it, and while you do have to pay money to show up to
> face-to-face meetings, it has otherwise been "free" to
> contribute and get the benefits of access to and review by
> many people for your ideas - even more so if they get adopted
> by a working group and published.  It seems fair to expect you
> to take on a risk of "jury duty" as part of your "citizenship".
> 
> Because I agree with John - right now we only have a
> distinctly smaller group of people volunteering than those
> contributing, and that does skew the demographic that the
> random number get to pick from.  We should strongly discourage
> not being in that pool.

Bron, 

Not sure it is practical today but it is probably useful that I
share a legend with you.  Like almost all good legends, many of
its elements are true or were true once.  And this legend has
the interesting property that much of the IETF's model for its
leadership and how they were chosen is derived from it, however
different the world today is from the one of the legend.

Once there were an organization called the IETF.  It was more of
an engineering body than a standards one.  People attended, and
organizations supported their attendance, because the Internet
was still fairly new and evolving rapidly and because those
organizations felt a need to be supportive of the Internet and,
with it, the IETF.  Proposed Standards were recognized as
specifications of ideas that, despite best efforts, might not
actually work in practice and the technical term for a company
that deployed a Proposed Standard (much less a draft that was
expected to lead to one) to customers and at large scale
involved synonyms for the word "fool".  In that IETF, almost
everyone who was actively participating knew almost everyone
else who was actively participating and there was a great deal
of sense of mutual and collective responsibility.  Because it
was an obligation to the community, just about everyone who
could volunteered for the Nomcom and the mechanism for random
selection from the pool acted largely to assure that the same
people wouldn't be stuck with the job too regularly.  Once a
Nomcom was selected, it evaluated candidates (both incumbent who
were willing to serve again and alternatives) based on the
knowledge of the voting members of those candidates.  The Nomcom
didn't need to sit around waiting for volunteers for IESG and
IAB positions (in the legend, that is all there were), it was
expected to go out and recruit.  There were no questionnaires or
the like because, if a significant fraction of the Nomcom didn't
know a candidate, that candidate was presumed to be not part of
the community enough and hence to be inappropriate for a
leadership position.  Similarly, there were few if any
candidates who really wanted to be on the IAB or IESG and fewer
still who saw those positions as important career moves.  Rather
those positions were responsibilities one might be obligated to
take on for a few years in the best interests of the community.
Especially on the IESG, people served for a term or two in
steering, coordination, and consensus-evaluating roles and then
went back to doing substantive work in the IETF.  And, because
they were almost all people who had other responsibilities than
serving in the IESG (or IAB) there were tremendous incentives to
to keep the IESG/IAB roles narrow (typically commitments of no
more than one day a week) and avoid both responsibility or
authority creep and chartering WGs that were not going anywhere
(or keeping them going after it became clear that they were not
going to make substantive contributions is a reasonable period
of time(.

Now, there are many ways in which the IETF of today doesn't
match the legend.  Some of them are probably even improvements.
But, where we have procedures or assumptions that are based on
parts of the legend from which reality has diverged, it seems to
me that we either need to figure out how to change the reality
--which may (often) be impossible-- or we need to go back and
look at the relationship between the legend and our procedures
rather than simply tweaking procedures while hoping that the
important elements of the legend will still work out.

best,
   john

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 06 Aug 2020 16:19:31 +1000

On Thu, Aug 6, 2020, at 01:27, Alissa Cooper wrote:
> All good points, and I wonder whether it might be simpler to drop the 
> “has been appointed to a formal role” bit. 

I think that would be best, for all the reasons you describe.

---

From: Peter Koch <[email-masked]>
Date: Thu, 6 Aug 2020 12:12:04 +0200

> Path 5: Has been a listed author of at least 2 IETF stream RFCs within the last 5 years.  A draft that has been approved by the IESG and is in the RFC Editor queue counts.

I would like to support this proposal (including the scope narrowing for
Phase 4). It seems to me to perfectly fit the "good enough" threshold.
As a one-off solution, it is sufficiently resistant to gaming.
Also, it is a no worse approximation of "has been exposed to IETF culture and
individuals enough to be able to judge social interaction, technical and role specific
leadership skills" than what we have been using so far.

-Peter

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 6 Aug 2020 13:12:12 +0000

>    If there are no criteria, then there is no basis for challenge. 

The RFC doesn't require a basis, just an email.  "Rich doesn't understand how the IETF works" suffices.

No it doesn't, and I wasn't asking for that.  Sorry if I annoyed you.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 6 Aug 2020 16:09:32 +0200

Hi Alissa, hi all,

+1 on dropping “has been appointed to a formal role” as formal roles are too diverse. Otherwise this sounds good to me as an experiment for the next nomcom cycle. 

I have one more question: How do you define attendance? Is that has registered and logged into at least one session for any fraction of time?

I would actually be interested on some numbers before we make a final call. I specifically wonder how many people we have that are eligible by one of the paths 2-5 who are not already eligible by path 1?

My expectation would be that the more online meeting we have (where we count people as well that have been remote-only previously) the smaller the number gets of people who are not covered by path 1. Or maybe we should look at those numbers for the past meetings by taking remote attendees into account for 106?

Mirja

> On 5. Aug 2020, at 17:27, Alissa Cooper <[email-masked]> wrote:
> 
>

---

From: "Salz, Rich" <[email-masked]>
Date: Thu, 6 Aug 2020 14:18:30 +0000

I can also live with the draft after the "appointed to a formal role" is removed.

---

From: Ted Hardie <[email-masked]>
Date: Thu, 6 Aug 2020 08:42:17 -0700

Dear John,

It always helps if legends start with something like "once upon time" or
"when Noah was building his boat", to set the stage for what follows.  In
the legend below, I think you mean "when RFC 1602
<https://tools.ietf.org/html/rfc1602>was new", based on this section from
it:

      Any member of the Internet community with the time and interest is
      urged to participate actively in one or more IETF Working Groups
      and to attend IETF meetings.  In many cases, active Working Group
      participation is possible through email alone; furthermore,
      Internet video conferencing is being used experimentally to allow
      remote participation.  Participation is by individual technical
      contributors rather than formal representatives of organizations.
      The process works because the IETF Working Groups display a spirit
      of cooperation as well as a high degree of technical maturity;
      IETF participants recognize that the greatest benefit for all
      members of the Internet community results from cooperative
      development of technically superior protocols and services.

      Members of the IESG and IAB are nominated for two-year terms by a
      committee that is drawn from the roll of recent participation in
      the IETF and chartered by the ISOC Board of Trustees.  The
      appointment of IESG and of IAB members are made from these
      nominations by the IAB and by the ISOC Board of Trustees,
      respectively.

Other, later descriptions, formalize the name of the committee and a lot
else, but this seems to be the era we're looking back on.  If you meant
this to be a reference to an earlier time, I'd appreciate knowing when.

The issue is that in 1994 there were already more than a thousand people
coming to some IETF meetings (San Jose, IETF 31 had 1079) and the
proceedings suggest the working groups were already split across many
different activities <https://www.ietf.org/proceedings/31/>.  So the
chances that any particular individual knew everyone else coming to the
IETF was vanishingly small and the work of people like Robin Dunbar (of the
famous "Dunbar's Number <https://en.wikipedia.org/wiki/Dunbar%27s_number>")
suggests it won't help much even if they had met them in passing.  Instead,
it's much more likely that a small group of highly visible people (e.g.
those already in leadership) were those who are sufficiently well-known to
be the targets of leadership requests in the outreach form you describe,
especially if it required the entire set of a committee to be familiar with
them.

That fosters a fundamentally conservative, go-with-who-you-know approach
that tends to see the same faces recycled again and again.  Since both of
us have served as WG chairs, IESG members, IAB members, and appointed
document editors, I think we might have to agree that this is the case in
the IETF, even after the era in which volunteering for the positions
started to be common.

The concerns I expressed were with layering on top of that already
conservative approach an additional shift to membership of the nomcom.  The
shift to a "known-to-the-community" metric rather than a "knows the
community" metric would, in my opinion, reinforce the likelihood that those
leadership positions would be assigned within relatively closed networks.
I think that's a bad idea. Even the appearance of it discourages
participation, and the reality of it could make the IETF completely
unwilling to take in new perspectives and new participants.  If we want the
IETF to be working well in a decade or a few decades time, we need to be
doing what we can to open up, rather than close down.

Just my opinion, of course,

Ted Hardie

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 7 Aug 2020 09:01:47 +1200

> I have one more question: How do you define attendance? Is that has registered and logged into at least one session for any fraction of time?

That would be my preference, but I don't know if we have that in a database. We do have "registered" in a database for 107, and "registered and paid/waived" in a database for 108.
 
> I would actually be interested on some numbers before we make a final call. I specifically wonder how many people we have that are eligible by one of the paths 2-5 who are not already eligible by p…

Yes, we should certainly regenerate the Venn diagrams for the revised criteria. That is real work, as is converting the results to SVG.RFC7996, so it will only happen when we have a stable version of the criteria.
 
> My expectation would be that the more online meeting we have (where we count people as well that have been remote-only previously) the smaller the number gets of people who are not covered by path…

I'm not sure that would be a useful comparison, because for 106 remote attendance was a very different thing in terms of both cost and commitment. Also remember that COVID-19 is not the only motivation for this discussion. People have been complaining for many years that remote participants get a bad deal.

Regards
   Brian

---

From: Ted Lemon <[email-masked]>
Date: Thu, 6 Aug 2020 17:07:08 -0400

FWIW, I’m fairly sure we have remote attendee blue sheets for 106, 107 and 108. They were not collected automatically until 108, so there would be some work required to get them for 106 and 107, but it is almost certainly possible if we care to do so.

> On Aug 6, 2020, at 5:01 PM, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 7 Aug 2020 09:21:44 +1200

On 06-Aug-20 16:55, Bron Gondwana wrote:
...
> Because I agree with John - right now we only have a distinctly smaller group of people volunteering than those contributing, and that does skew the demographic that the random number get to pick from.  

Not ignoring John's legend at all; in fact I vaguely remember those days lost in the mists of time. But the skew is towards people ready to spend time and mental energy on the work of the NomCom. (They may be willing for altruistic reasons, or because their boss told them to do it.)

> We should strongly discourage not being in that pool.

By switching to an opt-out model, I think we would very likely get a very annoying unintended consequence. A significant fraction of the unlucky 10 selected at random would have failed to opt out but would refuse to serve, or would simply ignore all the messages telling them that they had been selected. If you remember the good old days when we used to have face to face plenaries, you will also remember that only about half the attendees showed up. The others are people who are simply not interested in such matters (and they are definitely not on this list). I know because I've quite often asked people why they missed plenary. My wild guess is that at least 25% of people would fail to opt out and would refuse to serve if picked.

Regards
   Brian

---

From: "Bron Gondwana" <[email-masked]>
Date: Fri, 07 Aug 2020 11:29:06 +1000

> By switching to an opt-out model, I think we would very likely get a very annoying unintended consequence. A significant fraction of the unlucky 10 selected at random would have failed to opt out b…

Yes, that's fair.  This is a hard problem :(

I guess the better question is - is there anything we can do culturally or structurally to encourage people to volunteer (or indeed discourage them from not volunteering)?

Bron.

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Aug 2020 01:07:30 -0400

Hi Ted,

--On Thursday, August 6, 2020 08:42 -0700 Ted Hardie
<[email-masked]> wrote:

> Dear John,
> 
> It always helps if legends start with something like "once
> upon time" or "when Noah was building his boat", to set the
> stage for what follows.

My first draft contained "once upon a time" but I dropped it for
two reasons: (1) In recent years, I've often been attacked for
using too many words and having said "legend", they didn't seem
necessary to me.  (w) More important, and more to your main
point, I'd not prepared to argue that all of the elements of
that legend were ever true, especially all at the same time.  At
the same time, I will defend the assertion that our current
model for selecting leadership and the assumptions about who
would end up in the leadership and how they would interact with
the community assumed key elements of that legend... and that
was the reason for telling the story.  More below.

>  In the legend below, I think you mean
> "when RFC 1602
> <https://tools.ietf.org/html/rfc1602>was new", based on this
> section from it:
> 
>       Any member of the Internet community with the time and
> interest is       urged to participate actively in one or more
> IETF Working Groups       and to attend IETF meetings.  In
> many cases, active Working Group       participation is
> possible through email alone; furthermore,       Internet
> video conferencing is being used experimentally to allow
>       contributors rather than formal representatives of
> organizations.       The process works because the IETF
> Working Groups display a spirit       of cooperation as well
> as a high degree of technical maturity;       IETF
> participants recognize that the greatest benefit for all

It may also be worth noting that most of the above paragraph
comes from RFC 1310.  See below for more about that.

>       Members of the IESG and IAB are nominated for two-year
> terms by a       committee that is drawn from the roll of
> recent participation in       the IETF and chartered by the
> ISOC Board of Trustees.  The       appointment of IESG and of
> IAB members are made from these       nominations by the IAB
> and by the ISOC Board of Trustees,       respectively.
> 
> Other, later descriptions, formalize the name of the committee
> and a lot else, but this seems to be the era we're looking
> back on.  If you meant this to be a reference to an earlier
> time, I'd appreciate knowing when.

On the one hand, yes, 1602 reflects important elements of the
legend but the match between the legend and reality wasn't very
good even then.  For example...

> The issue is that in 1994 there were already more than a
> thousand people coming to some IETF meetings (San Jose, IETF
> 31 had 1079) and the proceedings suggest the working groups
> were already split across many different activities
> <https://www.ietf.org/proceedings/31/>.  So the chances that
> any particular individual knew everyone else coming to the
> IETF was vanishingly small and the work of people like Robin
> Dunbar (of the famous "Dunbar's Number
> <https://en.wikipedia.org/wiki/Dunbar%27s_number>") suggests
> it won't help much even if they had met them in passing.

Yes, and see above.    The nice thing about legends is that they
are not bound to specific facts and details.  Sometimes factual
evidence that contradicts them doesn't cause them to go away.

> Instead, it's much more likely that a small group of highly
> visible people (e.g. those already in leadership) were those
> who are sufficiently well-known to be the targets of
> leadership requests in the outreach form you describe,
> especially if it required the entire set of a committee to be
> familiar with them.

There you are getting to an assertion that my telling of the
legend didn't claim.  While I probably didn't say anything that
would be inconsistent with the "entire set of a committee" being
familiar, the condition for that part of the legendary model
working isn't that every member of the committee be familiar
with every plausible leadership candidate.  It is only that
there be some reasonable level of trust between members of the
committee (and acquaintance and familiarity among them even if
it post-dates their appointment) and that there be a contact
network so that members of the potential leadership group be
separated from some members of the committee by typically one
link and at most two, and hence from all members of the
committee by two and at most three links/degrees (with even more
twos than one would expect from the individual connections
because of non-committee interconnections within the community).
I'm not sure I'm explaining that very well, but the point is
that contact network familiarity rather quickly generates a much
larger group than one would get from Dunbar's work, especially
if one applied his work to the intersection of the connections
of individual committee members.

> That fosters a fundamentally conservative,
> go-with-who-you-know approach that tends to see the same faces
> recycled again and again.  Since both of us have served as WG
> chairs, IESG members, IAB members, and appointed document
> editors, I think we might have to agree that this is the case
> in the IETF, even after the era in which volunteering for the
> positions started to be common.

That is a different issue and, with one small qualification, I
agree with you now, agreed with you long before I wrote my note
to Bron, and tried to say so in a yet earlier note.  Like many
legends, it assumes a certain level of cultural homogeneity: I
assume that few people in the IETf whose family histories
involve European backgrounds remember the famous legend of Snow
White, the four dwarves, the giant, and two fairy princesses.
But, for better or worse, neither that legend, not the processes
derived from it, paid a lot of attention to diversity or
inclusiveness.   

The qualification is that one can take exactly the same pattern
of symptoms and conclude either "conservative,
go-with-who-you-know" or that we were able to do those jobs in
sequence and that this is some sort of meritocracy or, for that
matter, lack of willing people to take on those roles (or at
least ones who would not be worse).  Distinguishing among those
three hypotheses, or sorting out what combination of them a
given appointment represents, requires more data than the simple
pattern of appointments.

In addition, while I contend that legend strongly influenced the
thinking of the community when RFC 1602 (and the later
2026/2027) were written and an examination of the
(pre-Kobe-reforms) RFC 1310 shows little or no trace of it, my
recollection of the practices around the time of RFC 1310 is
that the IAB appointed the IETF Chairman (sic) and the IETF
Chair designated the Areas and appointed the Area Directors.
However much the legend leans toward a go-with-who-you-know
model, appointments by one person who is, himself (definitely
"him" at the time despite a few women on the early IAB(s))
appointed lean even more in that direction.   So the legend, and
the plan at the time of RFC 1602, where, in that sense, a step
forward.
 
> The concerns I expressed were with layering on top of that
> already conservative approach an additional shift to
> membership of the nomcom.  The shift to a
> "known-to-the-community" metric rather than a "knows the
> community" metric would, in my opinion, reinforce the
> likelihood that those leadership positions would be assigned
> within relatively closed networks.  I think that's a bad idea.

Agreed.  Strongly.  

> Even the appearance of it discourages participation, and the
> reality of it could make the IETF completely unwilling to take
> in new perspectives and new participants.  If we want the IETF
> to be working well in a decade or a few decades time, we need
> to be doing what we can to open up, rather than close down.

Yes.  Again, I don't think we disagree about the above in any
significant way.   My purpose in telling the "legend" story was
to point out that, regardless of how well that legend matched
the reality of the IETF at the time it came into being, it has
become more of a mismatch in recent years.  To the extent to
which our historical procedures for selecting our leadership and
the people who do those selections are based on that legend and
impede our getting an equally or more qualified and more diverse
leadership, then we need to fix that in more fundamental ways
than small incremental tinkering with the Nomcom eligibility
formula.  And, while I can see the case for making small changes
first, I think it would be a big mistake to use such small
changes (whether as an "experiment" or not) as an excuse to push
examining those larger issues a year or more into the future.

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Fri, 07 Aug 2020 01:31:02 -0400

--On Friday, August 7, 2020 11:29 +1000 Bron Gondwana
<[email-masked]> wrote:

>> By switching to an opt-out model, I think we would very
>> likely get a very annoying unintended consequence. A
>> significant fraction of the unlucky 10 selected at random
>> would have failed to opt out but would refuse to serve, or
>> would simply ignore all the messages telling them that they
>> had been selected. If you remember the good old days when we
>> used to have face to face plenaries, you will also remember
>> that only about half the attendees showed up. The others are
>> people who are simply not interested in such matters (and
>> they are definitely not on this list). I know because I've
>> quite often asked people why they missed plenary. My wild
>> guess is that at least 25% of people would fail to opt out
>> and would refuse to serve if picked.
> 
> I guess the better question is - is there anything we can do
> culturally or structurally to encourage people to volunteer
> (or indeed discourage them from not volunteering)?

I think that is one of the very interesting questions and I
think we actually know a possible mechanism, albeit one that has
its own side effects.  There is probably no way to prove it, but
I think it is a reasonable hypothesis that one of the things
that prevents people from volunteering is the very significant
time commitment involved with a Nomcom that has to select as
many people as now required.  In the hypothetical time of the
legend, the IESG was smaller, there were no Trust or LLC seats
to fill, and, since levels of knowledge by Nomcom members were
higher, presumably less time was needed to design and review
questionnaires, conduct candidate interviews and share knowledge
about them with other Nomcom members, and to request and take
advise from liaisons (and advisors?) about what skills positions
actually required (rather than, e.g., having liaisons act mostly
as process integrity monitors.

In theory at least we could split the Nomcom's responsibilities
up into two groups, say IESG and Trustees in one and IAB and and
LLC and anything else I've forgotten in the other.  That would
presumably trim the workload and increase the number of people
willing to volunteer (although I can't guess by how much).  Of
course, it would mean two chairs, two sets of liaisons, etc.  It
would effective prevent the Nomcom from having a debate as to
whether someone would be better suited for the IESG or IAB (or
some other role) but that might be a good thing in some ways.
And, if it were something we really wanted to explore, it,
unlike a few of the other things that have been discussed, would
be plausible as an experiment for a year or two.

   john

---

From: Alissa Cooper <[email-masked]>
Date: Fri, 7 Aug 2020 16:23:00 -0400

As I said in my previous email, we have attendance records for both in-person and remote attendees for all of the meetings affected by the proposals made thus far. Someone is considered a remote “attendee” if they logged in to at least one session on Meetecho.

Alissa

> On Aug 6, 2020, at 5:07 PM, Ted Lemon <[email-masked]> wrote:
>

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 16:55:27 -0400

> I work in security.  Let's try to estimate what is the worst that could
    > happen, and see how much effort we need to expend to prevent that from
    > happening.

    > On the plus side, we'd be more inclusive and get fresh voices.  On the
    > minus side, we might have some folks drop out. We might get some bad
    > choices for AD, etc., but we've weathered those before, and it's not

I disagree with your analysis here.

**They'd have to randomly selected from the pool into the nomcom**

Then, they'd have to either have the numbers or the influence to send nomcom
in a bad direction.   That's a quite a steep hill.
So the most likely choice is that the nomcom chair would have a harder time
of it.

    > clear that we'd get more bad AD's, etc., standing for the position than
    > we usually do.

Yes, that's particularly true since the IAB won't confirm anyone as an AD who
hasn't been a WG chair before.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 17:00:22 -0400

Joel M. Halpern <[email-masked]> wrote:
    > members to volunteer for nomcom to make a change in the way the IETF works to
    > better meet their needs.  Whether this was a bank industry association, a
    > government association, some competing SDO, or some open soruce alliance
    > doesn't matter.  It could easily produce results that are very bad for the
    > IETF.

So, my non-mathematically guess is that they'd need about 20x our current
pool size in order be sure to get at least 6 people into the voting set.
Our pool size is ~200, so they'd need 4000 people.
(Could they do it with only 1000? Maybe.  Can we agree that it's in O(10^3)?
I don't feel like doing the math on a Friday afternoon)

The simplest defense would be essentially be a CAPTCHA, perhaps not all
automated.

To do this right, on the first year, they'd just screw with IAB nominations,
because they need to pad the IAB with robots into order to get IESG
comfirmations the next year.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 17:15:01 -0400

Alissa Cooper <[email-masked]> wrote:
    > compromise that could be satisfactory that combines Ted’s proposal with
    > the other criteria listed in the draft and limits the experiment to one
    > year to minimize the threat of gaming. The resulting paths to
    > eligibility would look something like the following (using the draft’s
    > numbering):

I support this as a basis for 2021.
I share you concern for getting a document in front of the entire IETF sooner.

As many of the signals we are examining involve multiple years, it is
important to me that indicate what the trend will be for 2022/2023,etc.

What would it mean for the experiment to succeed? [would we expand further?]
What would it mean for the experiment to fail? [would we rescind one or all
of the new criteria?]
What would our evaluation criteria be?  I have some thoughts about how to
numerically (statistical survey) measure this.

In some sense, I'm specifically interested here in the IETF gaming the industry :-)
We want: more draft authors, more WG secretaries, more WG chairs.
(And more Document Shepherds, but that's not in our criteria, except that it
is a path towards bigger WG roles)

    > Path 1 requires attendance at 3 out of 5 meetings, Path 1 does not
    > create real avenues for gaming that I can see. People who wanted to
    > game the system would have needed to start already with IETF 108 or
    > 107, and it seems unlikely for that to have happened.

I agree with your comments here.
I believe that we are generally overstating the concern about gaming.

    > If people are interested in this, we could re-run the numbers and
    > diagrams by somehow incorporating 107 and 108 (e.g., selecting people
    > who attended three out of the last five meetings as of today, or two
    > out of the last three) to see what they produce.

Yes, I think that this would be possible.

I would suggest that we also do a run where we simulate 109 and 110
(replicating 108's numbers) and see what turns out for Spring 2021.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 17:22:35 -0400

> Unfortunately, 8713 also talks about how to replace nomcom members and
    > it says re-run the randomization.  I think we should not do that, but
    > have a list of “pinch hitters” available.

Generally, the nomcom chair knows who the 11th/12th/13th person is already.
One does run into selected people who turn out to be unable/unwilling to serve.

NOMCOM have discussed going back ten years the thought of running with the
11th person included, but not getting to vote.  As Ted says, it's a thankless
task.

Nomcom's have lost one person three or four times over twenty years, usually
to family/health related issues.

A broken nomcom is broken, it does not result in bogus ADs, because IAB
confirmation.  A broken nomcom probably just doesn't finish.

To screw really up the IETF, you need to slip three or four IAB people in,
two years running, and then you can attack the IESG.
Probably easier to just donate a ton of money to IETF LLC and/or ISOC.

---

From: "Salz, Rich" <[email-masked]>
Date: Fri, 7 Aug 2020 21:28:55 +0000

When I saw "I disagree" this is not the  conclusion I expected to see :)

    of it.

But that's convention, not requirement.

---

From: Michael StJohns <[email-masked]>
Date: Fri, 7 Aug 2020 17:42:48 -0400

> confirmation.  A broken nomcom probably just doesn't finish.

Actually -

Over the years there has been tension (and at times loud arguments) 
between the themes of  "The Nomcom selects the winner and the confirming 
body rubber stamps unless they see some process violation" and "The 
Nomcom selects the candidate, but the confirming body does its own due 
diligence and has no issues with rejecting the nomination if in its sole 
discretion the rejection is warranted".

Depending on the IAB, a "bad choice" for some value of "bad" could slip 
through if the IAB as a group follows the first theme rather than the 
second.

I've had more arguments than I care to recount about which of the two 
themes needs to be the correct theme.  If the Nomcom process (selection 
of members, selection of candidates) is weak or fraught, we get no 
protections if the confirming body follows the first theme.

Later, Mike

---

From: Michael StJohns <[email-masked]>
Date: Fri, 7 Aug 2020 18:20:36 -0400

> Our pool size is ~200, so they'd need 4000 people.

For "to be sure" ~= 99% the number is less than half of that.

Our pool size generally runs about 150 - 200 and has recently (ISTR) as 
low as 120-130.   The rough number for a good chance to get 6 people in 
the voting set would be 60% of the pool or 300 people (which would be a 
pool size of 500 assuming 200 normal volunteers).

At that level (assuming completely disjoint and fake organizational 
associations), they would have a 63.3% chance of having 6 or more 
members, and 83.4% chance of having 5 or more members and a 94.5% chance 
of having 4 or more members.

If they capture 70% of the pool (466 out of 666), they have an 85% of 6 
or more members, 95.3% of 5 or more and 99% of 4 or more.

80% of the pool (800 of 1000) gives them 96.7/99.5/99.9

90% of the pool (1800 of 2000) gives them 99.8/99.9/100

With a pool size of 4000, they'd have 4000/4200  or 95% of the pool  and 
100% chance of 6 or more members with a 61% chance of having all 10.

(these are rounded numbers!!)

Note that the only figure of merit is share of the pool size - it 
doesn't matter how large or how small the total pool is - at least until 
you start getting very small pools.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 21:38:57 -0400

> between

I wish they could have been louder, because I don't think the community understands.

    > the themes of  "The Nomcom selects the winner and the confirming body rubber
    > stamps unless they see some process violation" and "The Nomcom selects the
    > candidate, but the confirming body does its own due diligence and has no
    > issues with rejecting the nomination if in its sole discretion the rejection
    > is warranted".

    > through if the IAB as a group follows the first theme rather than the second.

    > I've had more arguments than I care to recount about which of the two themes
    > needs to be the correct theme.  If the Nomcom process (selection of members,
    > selection of candidates) is weak or fraught, we get no protections if the
    > confirming body follows the first theme.

In 5 nomcoms, going back 17 years, I have never had an IAB operate on the
first theme.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 21:43:19 -0400

> For "to be sure" ~= 99% the number is less than half of that.

Thanks for doing the math.

    > If they capture 70% of the pool (466 out of 666), they have an 85% of 6 or
    > more members, 95.3% of 5 or more and 99% of 4 or more.

So, O(2.5) rather than O(3).
Close enough :-)

You've shown it's 1/5 as expensive as I've imagined.
I am still not entirely concerned.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 7 Aug 2020 22:37:47 -0400

I think your assumption that one needs 6 of the 10 in order to distort 
the nomcom process is overly optimistic.

Aside from that, put more generally, it would seem strange and 
unfortuante to give a large voice in running the IETF to arbitrary 
people who have not been involved in the IETF.  There are plenty of 
indivcations from other bodies that this can result in abrupt and 
unfortunte shifts in the nature of the organizations.

Yours,
Joel

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 8 Aug 2020 14:40:33 +1200

> In 5 nomcoms, going back 17 years, I have never had an IAB operate on the

I think that's normal. "Checks and Balances" has two components, and if you ask
a body to confirm decisions of another body, that implies some degree of checking.
Otherwise there is literally no point. If you want to limit the checking to
correct execution of the rules, you need to add that limitation to BCP 10.

I suppose the issue is that the confirming body and the NomCom can disagree
categorically, if NomCom says that X is the only acceptable candidate but
the confirming body says that X is unacceptable. The public isn't even aware
that any disagreement exists, and we have no rule to break the deadlock.

   Brian

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Aug 2020 23:38:55 -0400

Joel M. Halpern <[email-masked]> wrote:
    > I think your assumption that one needs 6 of the 10 in order to distort the
    > nomcom process is overly optimistic.

Well, I think one or two well prepared disruptive people could seriously mess
up the process, but I'm assuming 6 gets one majority.

---

From: Michael StJohns <[email-masked]>
Date: Sun, 9 Aug 2020 18:17:40 -0400

This is actually a reply to both of you.

>> In 5 nomcoms, going back 17 years, I have never had an IAB operate on the

How would you know?  Absent a confirming body saying "we decline to 
confirm this candidate", the exterior view for Theme 1 is identical to 
Theme 2.

> that any disagreement exists, and we have no rule to break the deadlock.

Actually, the rule is pretty straightforward, but various Nomcoms have 
pushed back on it:  If the CB declines to accept a candidate for a 
position, that candidate is done and the Nomcom has to go back and find 
someone else.  Nomcom's have  - in at least one case very publicly ISTR 
- argued that they (Nomcom) were the only finders of fact and that the 
CB must have very good reasons and they really should be process ones 
and they must state those reasons for refusing to confirm a candidate 
and the Nomcom must be allowed to argue with the CB (or at least a 
subset of these were argued).    This was the dual of the one and only 
mediated dispute we've had where the CB asked the Nomcom for more data 
on candidates and the Nomcom refused.  The mediator found for the 
Nomcom, but I think that was incorrect: the CB should have just declined 
to confirm and rejected those for which they thought they had 
insufficient information.

Then there's the oft stated/claimed but mistaken belief that the CB must 
accept or reject the entire slate rather than individual candidates, or 
alternately, that the CB must return the slate without confirmation if 
it rejects any individual so the Nomcom can "move people around".  Both 
of these have obvious problems with respect to checks and balances.

In any event, continuing down that path, if the Nomcom, when faced with 
a rejection of a candidate, refuses to submit a replacement, or 
continues to submit the rejected candidate, then I believe the CB (and 
ideally the Past Chair) should notify the ISOC chair and ask for a new 
Nomcom to be seated.  It's not in the text, but this would be an obvious 
process failure without any other obvious ways to break the stalemate 
without completely removing the checks and balances.

We have no rules for holding over for positions which become vacant at 
the end of the First meeting, but for which the Nomcom process 
(including confirmation) hasn't completed.  With the possible exception 
of the IETF chair and the definite exception of the LLC board and Trust, 
I don't think leaving positions on the IAB or IESG open until they can 
be filled is a critical failure even if it affects work throughput.   
I'd like to see language that permits the LLC Board and Trust to hold 
over until replace. I'd also like to see some language describing who 
acts as IETF chair when the position is vacant.

Ok - that ran longer than I was planning.

Mike

---

From: S Moonesamy <[email-masked]>
Date: Tue, 11 Aug 2020 16:06:27 -0700

Hi Brian,

I'll comment below.

>So I am about 180 degrees away from Rich on this proposal. I couldn't

It is unlikely that the people (it is likely less in magnitude) would 
fulfill basic criteria as the time window is not one meeting only.  I 
discussed this previously within the context of the criteria for 
"recall" eligibility.

The last NomCom controversy could be interpreted in several ways:

    (a) There are several entities optimizing for the existing criteria.

    (b) The person(s) has very little knowledge or understanding of IETF
        process or are not interested in that.

Is the social structure based on the assumption that conflict of 
interest does not exist?  Is there an assumption that anyone asking a 
question about NomCom is an enemy unless he/she is/was an Area 
Director/IAB member, etc.

I doubt that NomCom members over the past years have much knowledge 
about what is required for a LLC Director or IETF Trustee 
position?  That would likely apply to me if you asked me a question 
about that a few years or more ago.

Regards,
S. Moonesamy