Subject: Remote participation in interims -- experiences

From: Mark Nottingham <[email-masked]>
Date: Thu, 9 Feb 2017 19:48:07 +1100

Hi everyone,

We had a QUIC Interim meeting a couple of weeks back, supporting remote participants through WebEx. I thought it might be useful to summarise how we did it and what the feedback was, for other WGs' possible benefit. Feel free to ignore.

The meeting was hosted at Akamai's office in Tokyo, which I think is a fairly typical tech company sales-oriented office in a major city. The meeting took place in a large training room.

## Room setup

We wanted to facilitate open discussion among participants. In the past, WGs I've been involved with have done this using a boardroom or U-shape setup, but we had 50 (!) people register for the Interim. So, we used a series of concentric "U" shapes, with the chairs at a table at the open end of the "U" and the projection screens behind them.

This seemed to work pretty well; people could see and hear each other easily, and we had active engagement from people in the "outer" tiers (the document editors and very active participants tended to sit in the "centre" tier).

One thing that really helped was the tables; the room had a large number of two-person table/desks on rollers that we could configure how we liked. That let us tweak the setup to get the right density and layout.

The room was quite large, so we tightened things up a bit and closed a divider wall to make the space smaller, so that it was easier to see and hear, both for local and remote participants.

## A/V

Other WGs Iv'e been involved in have run Interims with remote participation before, but the experience was generally pretty bad, and few people took us up on it. Those that did tended to be silent, and to leave the meeting early.

For this meeting, we had a sizeable number of people (15-20) register for remote participation, with several of them key participants (e.g., document editors). So, we had to provide a better experience.

Originally we wanted to use in-room videoconferencing (the room has multiple built-in cameras and an array of ceiling microphones), but that was a proprietary system that isn't easily available to non-employees.

Instead, we used:

- A dedicated computer: Having a dedicated computer was really helpful. I used an old MacBook Air; the only problem was that it only has two USB ports, meaning that we needed to unplug the camera to put a USB stick in to transfer a presentation. Next time, I'll bring a USB hub.

- Microphone: We used one of these: <http://www.mxlmics.com/microphones/web-conferencing/AC-404/> and it worked really well. It has an advertised range of 8m / 25ft, and the remote participants didn't seem to have any problem following conversation (once or twice they couldn't hear someone, but we also couldn't hear that person in the room - we just asked them to speak up). In a larger room, I'd probably use a pair of them and mux them in the OS (possible in both Windows and Mac with appropriate extensions / configuration). It cost me AU$147, I think it's available in the US for US$80.

- Camera: I used a <http://www.logitech.com/en-us/product/c922-pro-stream-webcam?crid=34>, but I'd actually prefer <http://www.logitech.com/en-us/product/12131?crid=34> due to its wider angle. Both do H.264 on-camera. The one I got came with a miniature tabletop tripod, which was very useful. Somewhere around AU$100.

- Speakers: We used the room's speakers (over HDMI, along with the projector); otherwise, I brought a pair of USB-powered speakers. A bluetooth speaker would also work, if the battery lasted long enough or if we plugged it in.

- WebEx: As provided through the Secretariat.

## Feedback

Afterwards, we ran a survey of remote participants to gather their feedback. The results are here:
  https://mnot.wufoo.com/reports/january-2017-quic-interim-feedback/
(please don't share this link widely; the people who responded were made aware that it would be shared with Chairs and IESG, but not further)

As you can see, people had somewhat conflicting experiences, but the overall feedback seems pretty positive, considering that remote participation is always going to be challenging (especially for a 3-day meeting).

## How we can Improve

My takeaways for making it better next time:

1) I'm not thrilled about buying another camera so soon after buying a new one (and that replaced a perfectly good one :), but many people mentioned the angle of view. I'm not sure how much video actually helps with remote participation, beyond establishing a "feel" of how the room looks (unless there's whiteboard work, obviously).

2) A tripod capable of being placed on the floor might help get a better camera angle.

3) A second USB mic will probably help, especially with more challenging audio environments. That said, the mic we chose is remarkably good.

4) Having a dedicated way to let a remote participant "raise their hand" -- even if it's just having someone watch Jabber -- would probably be good. That said, remote folks who had video didn't seem to have any problem with this, because WebEx "hears" when they're becoming active, and their window becomes prominent.

5) We could do a better job communicating the channels for remote participants and their details beforehand.

Hope this helps,

--
Mark Nottingham   https://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 10 Feb 2017 07:36:56 +1100

Link made public now, sorry!

> On 9 Feb 2017, at 7:48 pm, Mark Nottingham <[email-masked]> wrote:
> 
> Hi everyone,
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 13 Feb 2017 15:24:53 -0600

I was sitting in one of the U corners farthest from the chairs, and Mark's
summary matched my in-room experience. If I could add two points ...

I'm the responsible AD for QUIC, and having remote participation
consistently work as well as it seemed to work for this meeting makes me
feel much more comfortable with a working group that plans to do a
significant number of in-pdrson interims.

I've attended a fair number of interim meetings remotely, with a range of
experiences that were about as good as the meeting Mark described, all the
way down to "we ran speedtest this morning, and we're not sure the hotel
has enough bandwidth to let us monitor the jabber room", so I'm not sure
Jane Remoteattendee knows what to expect for any given meeting. Mark was
actively asking the remote addendees if they could see and hear (usually,
but not always, using jabber). Speaking only for myself, I value that level
of apparent interest, so that we don't have many awkward moments where we
poll the remote attendees and only then discover that they've all given up
hours ago!

Thanks,

Spencer

On Feb 9, 2017 02:48, "Mark Nottingham" <[email-masked]> wrote:

Hi everyone,

We had a QUIC Interim meeting a couple of weeks back, supporting remote
participants through WebEx. I thought it might be useful to summarise how
we did it and what the feedback was, for other WGs' possible benefit. Feel
free to ignore.

The meeting was hosted at Akamai's office in Tokyo, which I think is a
fairly typical tech company sales-oriented office in a major city. The
meeting took place in a large training room.

## Room setup

We wanted to facilitate open discussion among participants. In the past,
WGs I've been involved with have done this using a boardroom or U-shape
setup, but we had 50 (!) people register for the Interim. So, we used a
series of concentric "U" shapes, with the chairs at a table at the open end
of the "U" and the projection screens behind them.

This seemed to work pretty well; people could see and hear each other
easily, and we had active engagement from people in the "outer" tiers (the
document editors and very active participants tended to sit in the "centre"
tier).

One thing that really helped was the tables; the room had a large number of
two-person table/desks on rollers that we could configure how we liked.
That let us tweak the setup to get the right density and layout.

The room was quite large, so we tightened things up a bit and closed a
divider wall to make the space smaller, so that it was easier to see and
hear, both for local and remote participants.

## A/V

Other WGs Iv'e been involved in have run Interims with remote participation
before, but the experience was generally pretty bad, and few people took us
up on it. Those that did tended to be silent, and to leave the meeting
early.

For this meeting, we had a sizeable number of people (15-20) register for
remote participation, with several of them key participants (e.g., document
editors). So, we had to provide a better experience.

Originally we wanted to use in-room videoconferencing (the room has
multiple built-in cameras and an array of ceiling microphones), but that
was a proprietary system that isn't easily available to non-employees.

Instead, we used:

- A dedicated computer: Having a dedicated computer was really helpful. I
used an old MacBook Air; the only problem was that it only has two USB
ports, meaning that we needed to unplug the camera to put a USB stick in to
transfer a presentation. Next time, I'll bring a USB hub.

- Microphone: We used one of these: <http://www.mxlmics.com/
microphones/web-conferencing/AC-404/> and it worked really well. It has an
advertised range of 8m / 25ft, and the remote participants didn't seem to
have any problem following conversation (once or twice they couldn't hear
someone, but we also couldn't hear that person in the room - we just asked
them to speak up). In a larger room, I'd probably use a pair of them and
mux them in the OS (possible in both Windows and Mac with appropriate
extensions / configuration). It cost me AU$147, I think it's available in
the US for US$80.

- Camera: I used a <http://www.logitech.com/en-us/product/c922-pro-stream-
webcam?crid=34>, but I'd actually prefer <http://www.logitech.com/en-
us/product/12131?crid=34> due to its wider angle. Both do H.264 on-camera.
The one I got came with a miniature tabletop tripod, which was very useful.
Somewhere around AU$100.

- Speakers: We used the room's speakers (over HDMI, along with the
projector); otherwise, I brought a pair of USB-powered speakers. A
bluetooth speaker would also work, if the battery lasted long enough or if
we plugged it in.

## Feedback

Afterwards, we ran a survey of remote participants to gather their
feedback. The results are here:
  https://mnot.wufoo.com/reports/january-2017-quic-interim-feedback/
(please don't share this link widely; the people who responded were made
aware that it would be shared with Chairs and IESG, but not further)

As you can see, people had somewhat conflicting experiences, but the
overall feedback seems pretty positive, considering that remote
participation is always going to be challenging (especially for a 3-day
meeting).

## How we can Improve

1) I'm not thrilled about buying another camera so soon after buying a new
one (and that replaced a perfectly good one :), but many people mentioned
the angle of view. I'm not sure how much video actually helps with remote
participation, beyond establishing a "feel" of how the room looks (unless
there's whiteboard work, obviously).

2) A tripod capable of being placed on the floor might help get a better
camera angle.

3) A second USB mic will probably help, especially with more challenging
audio environments. That said, the mic we chose is remarkably good.

4) Having a dedicated way to let a remote participant "raise their hand" --
even if it's just having someone watch Jabber -- would probably be good.
That said, remote folks who had video didn't seem to have any problem with
this, because WebEx "hears" when they're becoming active, and their window
becomes prominent.

5) We could do a better job communicating the channels for remote
participants and their details beforehand.

Hope this helps,

---

From: Michael Richardson <[email-masked]>
Date: Tue, 14 Feb 2017 16:20:49 -0500

> - WebEx: As provided through the Secretariat.

Webex -- JAVA ws app?
Or the newer WEBRTC version (not available again this week, but working
really well two weeks ago...)

Did you have the benefit of having seen the room in question in advance?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 15 Feb 2017 08:33:57 +1100

We used traditional WebEx. 

I have presented talks in that room before; knowing its setup was one of the reasons we chose it. 

Cheers,

Sent from my iPhone

> On 15 Feb 2017, at 8:20 am, Michael Richardson <[email-masked]> wrote:
> 
> advance?
> 
> --