Subject: DISPATCH Virtual Meeting

From: Ben Campbell <[email-masked]>
Date: Mon, 23 Mar 2020 21:28:17 -0500

HI,

I was very skeptical that DISPATCH could have a useful virtual meeting. I was afraid that we would not get a cross-section of interests, and that only people with a vested interest in agenda items would attend.

I was mistaken. This was one of the best DISPATCH meetings we’ve had since I’ve been a chair.

We had very good attendance. Possibly more than we would have had in-person. Discussions overall went very well.

Some observations:

Patrick and I planned in advance how we would share the chair load. Patrick acted as emcee, and I handled queue management and only jumped in occasionally. We badgered people into volunteering to take notes and scribe jabbers in advance of the meeting. If we fell down at any of that, the crowd would quickly jump in to support us (especially when I got the queue out of order—lots of experienced IETFers paid close attention. They pay especially close attention when one screws up the times of other sessions that one mentions in one's chair slides. Not that I would know.)

People handled queuing very well. No one spoke out of turn. We had multiple presenters who were new at IETF participation, but they mostly handled it well.

As far as I could tell, people did not have a problem joining. Maybe that was because we started after close-of-business for most of the US. Or because we were 10m past the hour. Or maybe we were luckly.

There was a lot of interesting side-discussion in the jabber chat—to the point that I wonder if we can encourage more of it to make it into the webex. I know that gets logged, and people can review it, but I doubt it will be captured in institutional memory as well as the minuted stuff.

( BTW, we might should remind participants that it does get logged and presenters may review that log. People can get a bit more harsh in the chat than they might be at the mike.)

We did get the occasional problem when a person forgot to mute. Overall it wasn’t bad. We got an occasional side coversation, but the worst was when someone has the session running on multiple devices with live mikes. (The feedback from that is very…distinctive).

Part way through we were asked to introduce speakers by full name. I assume that was meant to include people in the discussion queue. When people get introduced by the queue manager, they don’t tend to state their names. It’s probably easier to just let the introducer do that.

In my opinion, this approach could work on a regular basis. I don’t mean to say we can dispense with in-person meetings altogether-- the fact many people knew each other may have been an important success factor.

I’m sure Patrick will have additional thoughts, and we welcome feedback from anyone else who attended.

Thanks!

Ben.

---

From: Patrick McManus <[email-masked]>
Date: Tue, 24 Mar 2020 10:22:26 -0400

Thanks Ben! It was an experience :)

I took a few notes on how it went so I thought I would share them as well -
Maybe these notes will help you with your virtual sessions.

There was good turnout - around 150 on the webex. The overall experience
reminded me more of my brief days as a high school radio DJ than as a
traditional meeting moderator. (Pete Resnick, you're on the air!) The lack
of video made this feeling even stronger. It was really hard to understand
the feeling in the room without any body language to look at so you might
want to ask more prompting questions to see if they get a response.

Muting is absolutely vital. Periods in which we didn't have cooperation on
that were very painful and this cyclically popped up during the 2 hours.
The secretariat took some proactive muting action (thank you!) and you can
reach them at [email-masked] to get help. DMs to offending microphones
were only semi effective and beware that if the secretariat selects mute
all you might lose your mic or the presenters (this appears to have
happened to us once, it was no big deal - but beware) too without warning.

We chose to have the chairs show the slides to keep the chaos of screen
changing down to a minimum. We designated one chair to show/advance the
slides and the other to track the queue and let folks know it was there
turn. I recommend this. We kept the queue to be only after the presentation
to simplify the process - and I think that was the right call. Back and
forth worked fine.

We failed to make sure everyone was introduced by their full names when
speaking. We have this issue in person too, of course, but it seemed easier
to forget in the faceless forum but is probably even more important to
remember.

The etherpad bluesheets worked OK - nearly the same number of entries as
webex participants. I put the etherpad link in the webex chat and made sure
to verbally bring it up several times. I noted on the top of the etherpad
that the bluesheets were at the bottom, after the minutes so folks loading
the page for that purpose found them easily.

Other than our presenters, we didn't really hear from many new folks.
Sometimes that's just the way it is, but I do like to help with that
process when I'm chairing a meeting and I fear that the audio-queue was
more intimidating than the regular in person one. Something to think about.
Some kind of hum replacement might at least provide a clue as to whether
everyone is on the same page or not.

In general folks were good about the webex chat vs the jabber chat and
there was no jabber relaying that had to be done. Ben did get us note and
jabber volunteers beforehand which I think was vitally important.

HTH. good luck everyone!

On Mon, Mar 23, 2020 at 10:28 PM Ben Campbell <[email-masked]> wrote:

> HI,
>
> I was very skeptical that DISPATCH could have a useful virtual meeting. I
> was afraid that we would not get a cross-section of interests, and that
> only people with a vested interest in agenda items would attend.
>
> We had very good attendance. Possibly more than we would have had
> in-person. Discussions overall went very well.
>
> Some observations:
>
> Patrick and I planned in advance how we would share the chair load.
> Patrick acted as emcee, and I handled queue management and only jumped in
> occasionally. We badgered people into volunteering to take notes and scribe
> jabbers in advance of the meeting. If we fell down at any of that, the
> crowd would quickly jump in to support us (especially when I got the queue
> out of order—lots of experienced IETFers paid close attention. They pay
> especially close attention when one screws up the times of other sessions
> that one mentions in one's chair slides. Not that I would know.)
>
> People handled queuing very well. No one spoke out of turn. We had
> multiple presenters who were new at IETF participation, but they mostly
> handled it well.
>
> As far as I could tell, people did not have a problem joining. Maybe that
> was because we started after close-of-business for most of the US. Or
> because we were 10m past the hour. Or maybe we were luckly.
>
> There was a lot of interesting side-discussion in the jabber chat—to the
> point that I wonder if we can encourage more of it to make it into the
> webex. I know that gets logged, and people can review it, but I doubt it
> will be captured in institutional memory as well as the minuted stuff.
>
> ( BTW, we might should remind participants that it does get logged and
> presenters may review that log. People can get a bit more harsh in the chat
> than they might be at the mike.)
>
> We did get the occasional problem when a person forgot to mute. Overall it
> wasn’t bad. We got an occasional side coversation, but the worst was when
> someone has the session running on multiple devices with live mikes. (The
> feedback from that is very…distinctive).
>
> Part way through we were asked to introduce speakers by full name. I
> assume that was meant to include people in the discussion queue. When
> people get introduced by the queue manager, they don’t tend to state their
> names. It’s probably easier to just let the introducer do that.
>
> In my opinion, this approach could work on a regular basis. I don’t mean
> to say we can dispense with in-person meetings altogether-- the fact many
> people knew each other may have been an important success factor.
>

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 24 Mar 2020 15:08:33 +0000

Hi,

>We failed to make sure everyone was introduced by their full names when speaking. We have this issue in person too, of course,
>but it seemed easier to forget in the faceless forum but is probably even more important to remember.

While I agree it is good to explicitly introduce the speakers, it was possible to see who was speaking by looking at the WebEx attendee list which shows where the audio comes from.

Regards,

Christer

On Mon, Mar 23, 2020 at 10:28 PM Ben Campbell <[email-masked]<mailto:[email-masked]>> wrote:
HI,

We had very good attendance. Possibly more than we would have had in-person.. Discussions overall went very well.

Some observations:

Thanks!

Ben.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 24 Mar 2020 10:12:52 -0500

> On Mar 24, 2020, at 10:08 AM, Christer Holmberg <[email-masked]> wrote:
> 
> Hi,
> 

I imagine that the note takers had enough to keep up with without having to constantly look at the WebEx GUI to see who is talking.

I personally found it difficult to tell who was speaking when looking at a list of oto 150 people, especially when it competes with the message pane for screen real estate. The tiles at the top help some, but the distinction between the current and recent speakers could be a bit more, well, distinctive.

Ben.

> 
> Regards,
> 
> Christer
> 
> 
> 
> On Mon, Mar 23, 2020 at 10:28 PM Ben Campbell <[email-masked] <mailto:[email-masked]>> wrote:
>> HI,
>>

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 24 Mar 2020 16:28:04 +0000

> I personally found it difficult to tell who was speaking when looking at a list of oto 150 people, especially when it competes with the message pane
> for screen real estate. The tiles at the top help some, but the distinction between the current and recent speakers could be a bit more, well, distinctive.

Yes. And, yesterday there weren't too many speakers to begin with. The situation would have been different if we e.g., would have discussed mandatory-to-impalement video codecs...

Regards,

Christer

 
On Mon, Mar 23, 2020 at 10:28 PM Ben Campbell <mailto:[email-masked]> wrote:
HI,

Some observations:

Thanks!

Ben.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 24 Mar 2020 14:17:41 -0400

Ben Campbell <[email-masked]> wrote:
    > There was a lot of interesting side-discussion in the jabber chat—to
    > the point that I wonder if we can encourage more of it to make it into
    > the webex. I know that gets logged, and people can review it, but I
    > doubt it will be captured in institutional memory as well as the
    > minuted stuff.

txauth restricted *webex* chat to mic line.
So it would be better if we could be consistent across WGs in how we use it.
And the webex chat is really hard to use, has no memory after you reconnect,
and jumps around everytime someone writes to it.

    > We did get the occasional problem when a person forgot to mute. Overall
    > it wasn’t bad. We got an occasional side coversation, but the worst was
    > when someone has the session running on multiple devices with live
    > mikes. (The feedback from that is very…distinctive).

The chair/host role can mute people.

I'm surprised you had the enter/exit stuff turned on.
I wouldn't have done that for any big group, or any situation where people
might have to reconnect.

Did you rely on the webrtc version?

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Ben Campbell <[email-masked]>
Date: Tue, 24 Mar 2020 13:21:52 -0500

> On Mar 24, 2020, at 1:17 PM, Michael Richardson <[email-masked]> wrote:
> 
> 

Sorry, I did not mean the webex chat. I meant “take it to the mike” :-)

> The chair/host role can mute people.

The chairs were not the WebEx host. The secretariat can and did mute people.

> I wouldn't have done that for any big group, or any situation where people

… again, not a chair decision.

> Did you rely on the webrtc version?

No, I used the MacOS local client.

---

From: "Salz, Rich" <[email-masked]>
Date: Tue, 24 Mar 2020 18:25:01 +0000

>    So it would be better if we could be consistent across WGs in how we use it.

We don't have enough data points to generalize yet.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 24 Mar 2020 18:22:17 -0400

> The chairs were not the WebEx host. The secretariat can and did mute people.

Why weren't the chairs the webex host?
I guess there can only be one host?

    > No, I used the MacOS local client.

I think that everyone should use the webrtc version if they can.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 24 Mar 2020 20:07:54 -0500

> On Mar 24, 2020, at 5:22 PM, Michael Richardson <[email-masked]> wrote:
> 
> 

That’s the way it was set up for us. I have no idea if there can be multiple hosts. I’m pretty sure a host can transfer the host role to someone else. But the chairs had enough on their hands chairing the meeting, without having to deal with the technical aspects of keeping the conference going. I think having the secretariat, or someone else, managing the bridge is the right way to do it; much like we have the MeetEcho staff manage remote participation for f2f meetings.

> I think that everyone should use the webrtc version if they can.

I know the local client usually works from my house. I have not found the WebRTC client as reliable. As chairs, our first concern was to try to make the meeting run smoothly.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 25 Mar 2020 12:38:03 +1100

> I imagine that the note takers had enough to keep up with without having to constantly look at the WebEx GUI to see who is talking. 

Speaking as a note taker, I didn't find it particularly hard - I have a 2-screen setup though, so I had webex up on one screen with the webex chat window open, and the speaker name appeared at the top of that. The hard part was people who had their "name" as something other than their actual name (Mark Nottingham as the QUIC working group, and some others who had email address rather than name) - but I could have easily noted what I saw there instead of the actual name and corrected to the name list later if I was struggling.

Where I missed something, others added it quickly too - having collaborative note taking is mostly nice (unless someone else starts taking ALL the notes like someone did in Tx, so I just left them to it).

Things I noticed:
 * Webex losing the slides or losing partial audio is a massive usability failure - if you don't know for sure that everyone is getting the same stream then you have the nightmare situation of "can everyone hear me / see" and needing 150 ACKs. Ouch.
 * I couldn't see where on Etherpad to coordinate the colours back to the identity of the person typing, which made it hard to contact them outside Etherpad to coordinate who's writing what - instead having to just write notes inline for the other authors!
 * There was a lot of interesting stuff happening in the Jabber chat which wasn't seen by those who weren't there. This makes the Webex-by-itself experience a significantly degraded understanding compared to Jabber. I feel like there would be some value in someone speaking a summary of what happened in Jabber to the Webex at the end of each presentation. It wouldn't take that long to get the summary out there - both key points made and a "feel of the room", e.g. in DISPATCH there was a large set of "the problem space should be solved, we don't think CLIENTID is the right solution" responses on Jabber, but those in Webex wouldn't have heard that sentiment so clearly. 

Overall I think this has gone really well given the short notice. Massive Kudos to both the secretariat and the rest of the IETF team for making it happen so smoothly, and to the chairs who really rose to the occasion and ran smooth remote meetings.

Cheers,

Bron.

--
 Bron Gondwana, CEO, Fastmail Pty Ltd
 [email-masked]

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Wed, 25 Mar 2020 03:10:02 +0000

> hi On Mar 24, 2020, at 12:29 PM, Christer Holmberg <christer.holmberg=[email-masked]> wrote:
> ...Yes. And, yesterday there weren't too many speakers to begin with. The situation would have been different if we e.g., would have discussed mandatory-to-impalement video codecs...

Mandatory impalement is highly controversial.

---

From: Ben Campbell <[email-masked]>
Date: Tue, 24 Mar 2020 22:14:15 -0500

> On Mar 24, 2020, at 10:10 PM, Mo Zanaty (mzanaty) <[email-masked]> wrote:
> 

Mo, you were there for the codec wars. You know what they are like :-)

(I really don’t know if Christer typoed that)

Ben.

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 25 Mar 2020 08:48:15 -0400

> On Mar 24, 2020, at 2:25 PM, Salz, Rich <rsalz=[email-masked]> wrote:
> 
> 

We have given the same instructions and setup to all the groups meeting this week, to try to have a more consistent experience for participants.

Alissa

> 
> _______________________________________________
> Manycouches mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/manycouches

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 25 Mar 2020 14:11:39 +0000

As note-taker in yesterday's ADD WG ...

From: Bron Gondwana

On Mar 24, 2020, at 10:08 AM, Christer Holmberg <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

I also used 2 screens and found it really easy to get the speakers' names from WebEx. Much easier (and with more accurate spelling) than transcribing the spoken form of names. It's really helpful if people make sure to provide WebEx with their full name. And chairs of other WGs should log out of their WG accounts before connecting to an interim meeting for a WG they aren't chairing - to avoid appearing in WebEx as "<WG name> working group".

Things I noticed:
*         Webex losing the slides or losing partial audio is a massive usability failure - if you don't know for sure that everyone is getting the same stream then you have the nightmare situation of "can everyone hear me / see" and needing 150 ACKs.  Ouch.
*         I couldn't see where on Etherpad to coordinate the colours back to the identity of the person typing, which made it hard to contact them outside Etherpad to coordinate who's writing what - instead having to just write notes inline for the other authors!
Since people usually don't bother identifying themselves in Etherpad, it probably wouldn't have made a difference, for you to have known that there is a button on the upper right of the Etherpad screen where you can enter your name (so people can see who that color is). I did think about identifying myself there. But I didn't, because it was mostly just me. I probably should have.
*         There was a lot of interesting stuff happening in the Jabber chat which wasn't seen by those who weren't there.  This makes the Webex-by-itself experience a significantly degraded understanding compared to Jabber.  I feel like there would be some value in someone speaking a summary of what happened in Jabber to the Webex at the end of each presentation.  It wouldn't take that long to get the summary out there - both key points made and a "feel of the room", e.g. in DISPATCH there was a large set of "the problem space should be solved, we don't think CLIENTID is the right solution" responses on Jabber, but those in Webex wouldn't have heard that sentiment so clearly.
I do sometimes put notes about jabber chatter into the minutes, when it seems particularly relevant. Even if it's just something like "<robust jabber conversation on this topic>". But I find it's also true in F2F IETF sessions that being in the jabber room (even when physically present) is an essential part of the experience.
Overall I think this has gone really well given the short notice..  Massive Kudos to both the secretariat and the rest of the IETF team for making it happen so smoothly, and to the chairs who really rose to the occasion and ran smooth remote meetings.

I also thought it went quite well.
I do have a comment for people who intend to speak (especially presenters) -
Consider doing whatever you can to ensure good quality audio is coming from you. For example, try to use wired Ethernet directly to your edge router (I moved my laptop and a monitor into the room where my router is so I could connect with Ethernet even though I wasn't speaking - because I didn't want to lose connectivity or have self-inflicted bad audio while taking minutes), use a headset that you've tested out and know works well, ask other family members to avoid bandwidth-consuming activities, etc.

---

From: "Pete Resnick" <[email-masked]>
Date: Wed, 25 Mar 2020 10:59:40 -0500

On 24 Mar 2020, at 10:12, Ben Campbell wrote:

> I personally found it difficult to tell who was speaking when looking 
> at a list of oto 150 people, especially when it competes with the 
> message pane for screen real estate. The tiles at the top help some, 
> but the distinction between the current and recent speakers could be a 
> bit more, well, distinctive.

There's a control in the upper right of the window which let's you 
select the view. The middle selection puts the current speaker in a pane 
above the list of participants. That was very helpful for me.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best

---

From: Dave Lawrence <[email-masked]>
Date: Wed, 25 Mar 2020 14:10:27 -0400

STARK, BARBARA H writes:
> As note-taker in yesterday?s ADD WG ...

On behalf of the ADD chairs, thank you again very much for your
service.  Note-taking is so often a thankless task, but we really do
appreciate what an excellent job you've done.

---

From: Michael Richardson <[email-masked]>
Date: Wed, 25 Mar 2020 19:46:16 -0400

> I know the local client usually works from my house. I have not found
    > the WebRTC client as reliable. As chairs, our first concern was to try
    > to make the meeting run smoothly.

So, to be clear, it's okay for the chairs to have a system that others can't use?

If the webrtc version doesn't work for you, then maybe we shouldn't use this
donated product, for which we have an unclear SLA.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 26 Mar 2020 00:58:25 +0100

Have been away for too long from webex hacking:

Is there a way to convert a webex URL into one which will only start the
webRTC client ? I am asking, because i think that the "normal" URL
makes it quite convoluted to find the webRTC URL, when i used it this
week, i had to wait for some while after the native client was offered,
before the "strt from browser" option would show up. Aka: Whoever makes
the marketing decision for webex these days seems to strongly prefer the
native client.

In zoom for example, its easy to convert URL: s|/j/|/wc/join/|

I thought it should be possile with webex too..

That at least would allow us to provide URLS both for native and webRTC
easier to participants.

-- 
---
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Thu, 26 Mar 2020 12:04:13 -0400

> webRTC client ? I am asking, because i think that the "normal" URL

No.
You need to logout of webex, or it will force the "full" client, or it will
tell you that you are not compatible.

Once logged out, do not accept the "desktop" application.
The IETF has put up with this BS for too long.
Clearly, open-stand is not important to us as users.

    > In zoom for example, its easy to convert URL: s|/j/|/wc/join/|

yes, and thanks for teaching that to me.   It should also be standard to use webrtc.