Subject: [Ietf108planning] FW: Preview of survey on virtual meetings

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Thu, 23 Apr 2020 12:19:01 +0000

Jay and others,

Please find below some comments on the survey.

"This survey is vital" isn't "vital" a little bit too much ?
"As we are asking f" better expand the "we" ?

Wow what a strange way to select a country... Can we simply list them? (I love to see Belgium at the top for once :-) )

In "Subscriber to an IETF mailing list within the last year" all other points use the past tense of a verb and not a noun.
In the same list, should we add "side meetings" ?

"How many IETF Meetings have you participated in, in total" is a little unclear, could we add 'physical and remote" ?

"If an in-person IETF meeting needs to be cancelled, should we hold a virtual IETF meeting instead?" should be qualified by a "one-week virtual IETF meeting" ?

Is it "prioritize" or "force a meeting" for newly formed WG? I suggest the latter

I had to stop after a while because way too many questions, I am afraid... 

-éric

-----Original Message-----
From: iesg <[email-masked]> on behalf of Jay Daley <[email-masked]>
Date: Thursday, 23 April 2020 at 02:27
To: The IESG <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Preview of survey on virtual meetings

    The small group set up by the IESG, LLC and IRTF Chair [1] to plan for IETF 108 has developed a survey to be sent out as widely as possible to IETF/IRTF participants to get views on the Future of Virtual Meetings.  You are now welcome to preview the survey [2] and provide feedback.  You can create a surveymonkey account and comment directly on the preview or send comments to [email-masked]

    We are hoping to send this survey out on Tuesday 28 April US time so please have any comments in well before that.  The survey is expected to run for ~9 days, less than is ideal but necessary given the possible work ahead.

    If you have any suggestions on distribution then those are also most welcome.

    thanks
    Jay

    [1]  Alissa Cooper, Colin Perkins, Martin Duke, Alvaro Retana, Alexa Morris, Greg Wood, Jay Daley
    [2]  https://www.surveymonkey.com/r/Preview/?sm=wlD1DqfXW8oxNST4ldDKk4dUN9OVHPs_2BCNWKZzQXuRAPHQA8Dy3EFy08_2B14_2BJekO#

    -- 
    Jay Daley
    IETF Executive Director
    [email-masked]

---

From: Warren Kumari <[email-masked]>
Date: Thu, 23 Apr 2020 09:39:29 -0400

> Wow what a strange way to select a country... Can we simply list them? (I love to see Belgium at the top for once :-) )

I agree that we really don't need this level of granularity -- a major
geography option seems better, just like we do for.... oh, bugger, I
didn't realize we capture this level of granularity for meetings --
https://datatracker.ietf.org/stats/meeting/106/country/
Oh, and give up on ever seeing Belgium at the top of any list, ever...
(ooooh! Burn!) (other than, perhaps, for a: chocolate and b: Hercule
Poirot)

I'm somewhat uncomfortable that this survey it too long / granular,
and that we are overcollecting / people will get bored midway through
and we bias towards hardcore attendees...

"Subscriber to an IETF mailing list within the last year"  -- this
feels like weird wording. I don't think I **subscribed** to any
mailing lists last year...

"Posted to an IETF mailing list within the last year", "Posted to an
IRTF mailing list within the last year" (and similar) -- do we need
this level of granularity? Could it instead be "Posted to an IETF or
IRTF mailing list within the last year" (and same for subscribe,
chair, etc).

"Did you participate in the one week virtual IETF 107 meeting (20-27
March 2020) that replaced the in-person Vancouver meeting?" -- what
does this **mean**? I listened to some session, was that
"participation"?

"How many other IETF Meetings have you participated in remotely
(excluding IETF 107)?", "How many IETF Meetings have you participated
in in-person?". Ok, but then is it really necessary to ask "How many
IETF Meetings have you participated in, in total?"

"Did the scheduling of the virtual interims since the one week virtual
IETF 107 meeting (20-27 March 2020) allow you to participate in all of
those you wanted to?" -- the "since" confuses me --- does this mean
the ones *after* the one week virtual (AKA, the other ones)?

"If an in-person IETF meeting needs to be cancelled, should we hold a
virtual IETF meeting instead?"  Yes! No! Maybe!!! I don't know how to
answer this without knowing what the eligibility implications, etc
are. Perhaps we can ignore this, but maybe including some weasel word
about "assuming we figure out the eligibility implications" ?

I don't know who added "Cookies", but whoever it was, thank you!

"Have each WG write 3 sentences to justify why they need to meet
during the meeting week rather than as an interim meeting" --- oooh,
interesting, but how do we evaluate the answers? Monkey knife fight?

I think that there are way too many time zones listed - we may need to
manually make a drop down if survey monkey doesn't have a ~24 item
list... Yes, we may miss UTC+12:45, but meh...

"How important is it to have the following functionality integrated
into the video conferencing tool? (Skip any you don’t know about)" --
perhaps add "integration"? So your datatracker login gets you in (this
isn't limited to Meetecho, we should be able to integrate something
from DT -> <webex/ zoom/ meet / whatever...)

"How important is it for the queue management tool to provide the
following features? (Skip any you don’t know about) " -- Hums?

"Is virtual hum technology needed?" -- Ah. It feels weird and
confusing to not have this on the previous page...

"How important is it to have a virtual hallway tool that replicates
the hallway environment?" s/replicates/attempts to replicate/ -- we
cannot really recreate the hallway environment. Aslo, perhaps add: do
you know about [email-masked]?

Phew!
W

> Ietf108planning mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf108planning

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Barry Leiba <[email-masked]>
Date: Thu, 23 Apr 2020 11:29:44 -0400

> https://datatracker.ietf.org/stats/meeting/106/country/

Even if we need the country, a simple alphabetical list of countries is best.
If there's some reason to separate them by continent, it's certainly
neither necessary nor useful to separate, say, Eastern Europe from
Northern Europe.

> and we bias towards hardcore attendees...

Exactly the comment I was going to make: this is way too long.  If we
do something like this, we *have to* keep track of how many people
bail out... how many answer some of the earlier questions and none of
the later ones.

> mailing lists last year...

This one, in particular, is likely to mislead, and the wording that's
there now is *better* than the suggested change.  We don't want to
know whether you created a *new* subscription, which is what I hear
from "subscribed to".  We want to know whether, during the last year,
you were a subscriber -- whether you follow discussions, which is hard
to judge, so "being subscribed" is a lame proxy for that.

> IRTF mailing list within the last year" (and same for subscribe,

I agree.  If we want to know about IETF and IRTF separately, we could
simply ask "participate in IETF working groups" and "participate in
IRTF research groups".  It's a lot harder to tease out levels of
participation, and I'm not sure to what extent it's useful to try.

> interesting, but how do we evaluate the answers? Monkey knife fight?

I went both ways on this when I read it, but in the end decided that
it has to be the responsible AD who decides, and then "discuss with
you responsible AD" works better for me than "each working group write
an essay."

> list... Yes, we may miss UTC+12:45, but meh...

Honestly, most of us are quite used to picking our time zones out of
long TZ lists anyway.  I don't really see that as a problem.

Barry

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 23 Apr 2020 17:40:43 +0200

I left a bunch of comments directly in the survey, especially about questions that I wasn’t sure how to answer because I didn’t fully understand them… but a few quick comments below.

> On 23. Apr 2020, at 17:29, Barry Leiba <[email-masked]> wrote:
> 

Agree. That distinction is weird. Don’t we usual just do the continent and not on country level. Do we need country level?
> bail out... how many answer some of the earlier questions and none of

Yes, I gave up midway of maybe 60-70% at the first look this morning.

> to judge, so "being subscribed" is a lame proxy for that.

Agree. I think there are also two questions merged here: on about the role and one about participation. Maybe we can split this up.

> participation, and I'm not sure to what extent it's useful to try.

There might be a point about understanding IRTF participation separately from IETF participation for the virtual schedule. But maybe better to split this up into two questions.

> an essay.”

Agree. Found this option also really weird and it doesn’t touch the real point about who decides. Left a similar comment.

> long TZ lists anyway.  I don't really see that as a problem.

This is the question where I gave up. Would prefer a different more simple question or questions.

---

From: Roman Danyliw <[email-masked]>
Date: Thu, 23 Apr 2020 16:02:55 +0000

Hi!

My de-duplicated list of feedback is:

Page 3 – the cookie option for “How important is it for each of the following to be included in a virtual IETF meeting?” is funny, but not needed

Page ? -- Assuming this applies to the worst possible timezone for you, how easy would it be for you to participate in each of the following virtual IETF meeting formats? (Skip any you don’t know about) – the visuals helped describe which days things were happening but the semantics of the vertical blue lines wasn’t intuitive to me.

Page ? -- Are you willing to participant more or less based on the registration fee.  Perhaps s/participate more/more likely to participate in the meeting/.  Maybe it was my incorrect read -- almost like "if I pay money I might participate in more/less in working group meetings".  It seems like it’s a meeting specific answer -- if you charge for the meetings, will you come to any of the meeting?

Roman

-----Original Message-----
From: Ietf108planning <[email-masked]> On Behalf Of Warren Kumari
Sent: Thursday, April 23, 2020 9:39 AM
To: Eric Vyncke (evyncke) <evyncke=[email-masked]>
Cc: [email-masked]
Subject: Re: [Ietf108planning] FW: Preview of survey on virtual meetings

On Thu, Apr 23, 2020 at 8:19 AM Eric Vyncke (evyncke) <evyncke=[email-masked]> wrote:
> Wow what a strange way to select a country... Can we simply list them? 
> (I love to see Belgium at the top for once :-) )

I agree that we really don't need this level of granularity -- a major geography option seems better, just like we do for.... oh, bugger, I didn't realize we capture this level of granularity for meetings -- https://datatracker.ietf.org/stats/meeting/106/country/
Oh, and give up on ever seeing Belgium at the top of any list, ever...
(ooooh! Burn!) (other than, perhaps, for a: chocolate and b: Hercule
Poirot)

I'm somewhat uncomfortable that this survey it too long / granular, and that we are overcollecting / people will get bored midway through and we bias towards hardcore attendees...

"Subscriber to an IETF mailing list within the last year"  -- this feels like weird wording. I don't think I **subscribed** to any mailing lists last year...

"Posted to an IETF mailing list within the last year", "Posted to an IRTF mailing list within the last year" (and similar) -- do we need this level of granularity? Could it instead be "Posted to an IETF or IRTF mailing list within the last year" (and same for subscribe, chair, etc).

"Did you participate in the one week virtual IETF 107 meeting (20-27 March 2020) that replaced the in-person Vancouver meeting?" -- what does this **mean**? I listened to some session, was that "participation"?

"How many other IETF Meetings have you participated in remotely (excluding IETF 107)?", "How many IETF Meetings have you participated in in-person?". Ok, but then is it really necessary to ask "How many IETF Meetings have you participated in, in total?"

"Did the scheduling of the virtual interims since the one week virtual IETF 107 meeting (20-27 March 2020) allow you to participate in all of those you wanted to?" -- the "since" confuses me --- does this mean the ones *after* the one week virtual (AKA, the other ones)?

"If an in-person IETF meeting needs to be cancelled, should we hold a virtual IETF meeting instead?"  Yes! No! Maybe!!! I don't know how to answer this without knowing what the eligibility implications, etc are. Perhaps we can ignore this, but maybe including some weasel word about "assuming we figure out the eligibility implications" ?

"Have each WG write 3 sentences to justify why they need to meet during the meeting week rather than as an interim meeting" --- oooh, interesting, but how do we evaluate the answers? Monkey knife fight?

I think that there are way too many time zones listed - we may need to manually make a drop down if survey monkey doesn't have a ~24 item list... Yes, we may miss UTC+12:45, but meh...

"How important is it to have the following functionality integrated into the video conferencing tool? (Skip any you don’t know about)" -- perhaps add "integration"? So your datatracker login gets you in (this isn't limited to Meetecho, we should be able to integrate something from DT -> <webex/ zoom/ meet / whatever...)

"How important is it for the queue management tool to provide the following features? (Skip any you don’t know about) " -- Hums?

"Is virtual hum technology needed?" -- Ah. It feels weird and confusing to not have this on the previous page...

"How important is it to have a virtual hallway tool that replicates the hallway environment?" s/replicates/attempts to replicate/ -- we cannot really recreate the hallway environment. Aslo, perhaps add: do you know about [email-masked]?

Phew!
W

>     The small group set up by the IESG, LLC and IRTF Chair [1] to plan 
> for IETF 108 has developed a survey to be sent out as widely as 
> possible to IETF/IRTF participants to get views on the Future of 
> Virtual Meetings.  You are now welcome to preview the survey [2] and 
> provide feedback.  You can create a surveymonkey account and comment 
> directly on the preview or send comments to [email-masked]
>
>     [2]  
> https://www.surveymonkey.com/r/Preview/?sm=wlD1DqfXW8oxNST4ldDKk4dUN9O
> VHPs_2BCNWKZzQXuRAPHQA8Dy3EFy08_2B14_2BJekO#
>
>
>     --
>     Jay Daley
>     IETF Executive Director
>     [email-masked]
>
>
> --

--
I don't think the execution is relevant when it was obviously a bad idea in the first place.
This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants.
   ---maf

---

From: Warren Kumari <[email-masked]>
Date: Thu, 23 Apr 2020 14:55:04 -0400

> to judge, so "being subscribed" is a lame proxy for that.

Sorry, I was unclear -- I wasn't suggesting new text, rather pointing
(apparently poorly!) out that this one is hard to parse / confusing...
Perhaps "Regularly read an IETF mailing list within the last year" is
clearer? Or follow an IETF mailing list?

Many people are subscribers to lists, but don't read them, etc. And,
if we just cared about # of subscribers (no idea why we would), we
could pull this from mailman...

> participation, and I'm not sure to what extent it's useful to try.

WFM.

---

From: Jay Daley <[email-masked]>
Date: Fri, 24 Apr 2020 07:57:34 +1200

Thanks all.

I’m still parsing the comments but some changes jump out at me (all of which need to discussed and agreed with the small group)

1. The country question is intended to help us assess how the breakdown of respondents by country looks in comparison to the same breakdown of previous meetings.  The strange format is to make it easier to analyse the responses after.  I suspect it’s best if we just lose that question because it potentially allows de-anonymisation and because the time zone question later on it much more important.  We did have a regional question there which we should probably revert to (the change to a country question was made at the last minute).

2.  The list of roles has grown too big as additional items have been added. All we really need to understand is some measure of how important that person is to the functioning of a meeting, and so I’ll look at paring it back to the minimum for that.

It seems that the combination of these two questions at the front and the way they are asked sets the wrong tone and frustrates people at the outset, which is never a good idea from the survey.

Jay

> On 24/04/2020, at 6:55 AM, Warren Kumari <[email-masked]> wrote:
> 
> On Thu, Apr 23, 2020 at 11:29 AM Barry Leiba <[email-masked] <mailto:[email-masked]>> wrote:
>> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/ietf108planning <https://www.ietf.org/mailman/listinfo/ietf108planning>
-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 23 Apr 2020 21:05:18 +0100

- Question numbers would've helped in this review.

- Why do we need such fine-grained location? (Svalbard:-)

- Which applies to you is always a bit irritating as it can
personally identify people (esp Colin I guess - that looks
a bit silly) We could coalesce a bunch of those with no
loss.

- A bunch of questions are quite complex relative to other
surveys. I don't know how that may affect results.

- "Did the scheduling of the virtual interims since the one
week virtual IETF 107 meeting (20-27 March 2020) allow you
to participate in all of those you wanted to?" seems like
an odd question - I'd expect 90% "no." Maybe better to ask
"on a scale of 1-5" or something.

- "how productive has your IETF meeting experience" could
confuse, because I wasn't "at" an IETF meeting. Maybe ask
instead "how productive has your IETF107 meeting-
replacement experience" instead?

- "virtual IETF meeting" - there were comments on the list
about that to the effect that "online IETF meeting" is
better. I'd say it may well be.

- All the "worst possible TZ" questions are a bit odd
because of that. Might be better to assume Madrid's TZ
for those questions, I think we'd get better info even
if we don't use the Madrid TZ in the end.

Cheers,
S.

PS: Longer term, it'd be nice if we had a less privacy
invasive service to use than survey monkey. In this case
I can't see comments from others without creating an
account (which I don't wanna do), and there's really no
need for user-tracking for our surveys.

On 23/04/2020 01:26, Jay Daley wrote:
> We are hoping to send this survey out on Tuesday 28 April US time so
> please have any comments in well before that.  The survey is expected
> to run for ~9 days, less than is ideal but necessary given the
> possible work ahead.
> 
> thanks Jay
> [1]  Alissa Cooper, Colin Perkins, Martin Duke, Alvaro Retana, Alexa
> Morris, Greg Wood, Jay Daley [2]
> https://www.surveymonkey.com/r/Preview/?sm=wlD1DqfXW8oxNST4ldDKk4dUN9OVHPs_2BCNWKZzQXuRAPHQA8Dy3EFy08_2B14_2BJekO#
>
> 
>

---

From: Jay Daley <[email-masked]>
Date: Fri, 24 Apr 2020 15:26:25 +1200

Stephen

> On 24/04/2020, at 8:05 AM, Stephen Farrell <[email-masked]> wrote:
> 
> 

Yes sorry, they get in the way when live but not for review.

> - Why do we need such fine-grained location? (Svalbard:-)

That has reverted to region.

> a bit silly) We could coalesce a bunch of those with no

It does need shortening -  a number of others have made similar comments.

> replacement experience" instead?

Done.

> better. I'd say it may well be.

I agree.

> if we don't use the Madrid TZ in the end.

Several people have said that.  We originally chose that and decided against it as we don’t want to pre-judge the timing of the meeting.  We will reconsider.

> need for user-tracking for our surveys.

The surveys are run with all user tracking turned off but I take your point about comments - others have made the same point.

Jay

> <0x5AB2FAF17B172BEA.asc>-- 
-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Jay Daley <[email-masked]>
Date: Fri, 24 Apr 2020 15:29:02 +1200

> On 24/04/2020, at 4:02 AM, Roman Danyliw <[email-masked]> wrote:
> 
> Hi!
> 

Thanks - We'll see what others say before trying to address that.

> Page ? -- Are you willing to participant more or less based on the registration fee.  Perhaps s/participate more/more likely to participate in the meeting/.  Maybe it was my incorrect read -- almos…

Changed that completely following the feedback to have actual fee levels (100%,75%,50%,25% of in-person) and ask "For each of these different sets of meeting registration fees, how likely is it that you will participate in an online IETF meeting? (skip any you don't know about)"

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Jay Daley <[email-masked]>
Date: Fri, 24 Apr 2020 15:33:05 +1200

> On 24/04/2020, at 12:19 AM, Eric Vyncke (evyncke) <evyncke=[email-masked]> wrote:
> 
> Jay and others,
> 

Not really - otherwise we’re just guessing.

> "As we are asking f" better expand the "we" ?

Will address that in the email that distributes the survey.

> Wow what a strange way to select a country... Can we simply list them? (I love to see Belgium at the top for once :-) )

Replaced with regions.

> In "Subscriber to an IETF mailing list within the last year" all other points use the past tense of a verb and not a noun.

That whole list needs paring down and that one can probably go.

> "How many IETF Meetings have you participated in, in total" is a little unclear, could we add 'physical and remote" ?

Tidied those up.

> "If an in-person IETF meeting needs to be cancelled, should we hold a virtual IETF meeting instead?" should be qualified by a "one-week virtual IETF meeting" ?

We don’t want to assume it will be one week.  I’ve adjusted the wording by adding "replaced with" to make that clearer.

> I had to stop after a while because way too many questions, I am afraid... 

All good.

thanks
Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 24 Apr 2020 10:16:36 +0100

Hiya,

All good otherwise, but just one follow-up:

On 24/04/2020 04:26, Jay Daley wrote:
>> PS: Longer term, it'd be nice if we had a less privacy invasive
>> service to use than survey monkey. In this case I can't see
>> comments from others without creating an account (which I don't
>> wanna do), and there's really no need for user-tracking for our
>> surveys.
> The surveys are run with all user tracking turned off but I take your
> point about comments - others have made the same point.
IIRC, when the survey goes live, ads are displayed, JS
is required and other web tracking galore happens, to
the point where I can't use my default browser setup to
take the survey. That's not a major deal, and happens
sadly often, but there's no reason why we need to use
an ad-driven survey service if a more privacy friendly
one exists.

I don't do surveys so not sure if there is such. But I
do run polls and always use dudle [1] rather than the
better known ad-driven alternative. Most people seem
to not know or care, a few like that, and I've only
ever had one person bemoan a missing feature from the
commercial service (google calendar integration). So
be worth someone checking if there's a good alternative
to survey monkey IMO (non-urgently).

Cheers,
S.

[1] https://dudle.inf.tu-dresden.de/better-than-survey-monkey/

---

From: Jay Daley <[email-masked]>
Date: Fri, 24 Apr 2020 21:37:58 +1200

> On 24/04/2020, at 9:16 PM, Stephen Farrell <[email-masked]> wrote:
> 
> 
> Hiya,
> 

I don’t know about the free tier of Surveymonkey, but there are no ads on the paid tiers and we are on a paid tier.

I’ve always found Surveymonkey quite clumsy and so at some point I do want to look for an alternative.

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 24 Apr 2020 16:54:52 +0000

Several folks have expressed that the survey is too long.  I agree that it is long, but most of the questions that are being asked seem useful, and hence I’m not sure what can easily be cut that would make a significant different to its length.  However, I expect that the length of the survey may skew the feedback towards individuals who are more involved with IETF.  Is this okay?

If the accompanying email makes it clear why the survey is slightly longer, and that the data will be shared, then I would hope that we would still get reasonable engagement from the active IETF community.

Rob

From: Ietf108planning <[email-masked]> On Behalf Of Jay Daley
Sent: 24 April 2020 04:33
To: Eric Vyncke (evyncke) <evyncke=[email-masked]>
Cc: [email-masked]
Subject: Re: [Ietf108planning] Preview of survey on virtual meetings

On 24/04/2020, at 12:19 AM, Eric Vyncke (evyncke) <evyncke=[email-masked]<mailto:evyncke=[email-masked]>> wrote:

Jay and others,

Replaced with regions.

Tidied those up.

All good.

thanks
Jay

-éric

-----Original Message-----
From: iesg <[email-masked]<mailto:[email-masked]>> on behalf of Jay Daley <[email-masked]<mailto:[email-masked]>>
Date: Thursday, 23 April 2020 at 02:27
To: The IESG <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Preview of survey on virtual meetings

   The small group set up by the IESG, LLC and IRTF Chair [1] to plan for IETF 108 has developed a survey to be sent out as widely as possible to IETF/IRTF participants to get views on the Future of Virtual Meetings.  You are now welcome to preview the survey [2] and provide feedback.  You can create a surveymonkey account and comment directly on the preview or send comments to [email-masked]<mailto:[email-masked]>

   thanks
   Jay

   [1]  Alissa Cooper, Colin Perkins, Martin Duke, Alvaro Retana, Alexa Morris, Greg Wood, Jay Daley
   [2]  https://www.surveymonkey.com/r/Preview/?sm=wlD1DqfXW8oxNST4ldDKk4dUN9OVHPs_2BCNWKZzQXuRAPHQA8Dy3EFy08_2B14_2BJekO#<https://www.surveymonkey.com/r/Preview/?sm=wlD1DqfXW8oxNST4ldDKk4dUN9OVHPs_2BCNWKZzQXuRAPHQA8Dy3EFy08_2B14_2BJekO>

   --
   Jay Daley
   IETF Executive Director
   [email-masked]<mailto:[email-masked]>

--
Ietf108planning mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ietf108planning

--
Jay Daley
IETF Executive Director
[email-masked]<mailto:[email-masked]>

---

From: Barry Leiba <[email-masked]>
Date: Fri, 24 Apr 2020 13:27:16 -0400

> Several folks have expressed that the survey is too long.  I agree that
> it is long, but most of the questions that are being asked seem useful,
> and hence I’m not sure what can easily be cut that would make a
> significant different to its length.  However, I expect that the length
> of the survey may skew the feedback towards individuals who are more
> involved with IETF.  Is this okay?

The problem is that I'm afraid that it will skew toward those who have
a particular desire to say something, and we need input from the
community as a whole, not a vocal few with, perhaps, axes to grind.
I'm afraid that even if we think that all the questions are important
and there's little we could cut without losing information, we will
lose more information because of frustration and people leaving the
survey in the middle than we would have by having a shorter survey in
the first place.

And, unfortunately, we won't know whether I'm right or not until after
the survey, when it's too late... so it comes down to your guess or
mine.  I think the best we can do is figure out not what information
we'd like to have, but what the minimum is that we can collect in
order to inform our planning reasonably.  Well, and avoidance of
dizzyingly involved questions that someone has to think more than a
few seconds to answer.

Barry