Subject: [113attendees] hybrid meetings: the worst of both worlds

From: Leif Johansson <[email-masked]>
Date: Fri, 25 Mar 2022 14:34:43 +0100

I spent the week onsite in Wienna...

As usual the arrangements are great and the local host made this a very nice experience.

And then there is the hybrid meeting thing... imo hybrid works well as technology and completely sucks on a human level.

This statement may be controversial and/or unpopular in the IETF where we're all about the tech but...

- hybrid means there is not enough folks onsite to create critical mass for the "hallway track"
- remote-participation is arguably better for the technical WG process than onsite at this point
- remote is really bad for the informal discussions (gather is very disappointing imo)

The most efficient WG were where most of the contributurs had decided to show up onsite.

The most important discussions I had were (as usual) not in a WG meeting.

Maybe the IETF needs to rehink the purpouse of onsite meetings.

We have made the remote experience so good that the "professionalized" aspect of churning out RFCs doesn't really need onsite.

However, an IETF with no human interaction might be professional but it won't be efficient.

	Cheers Leif

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 09:42:25 -0400

On 3/25/22 09:34, Leif Johansson wrote:
> And then there is the hybrid meeting thing... imo hybrid works well as 
> technology and completely sucks on a human level.
>

Always was my concern.  I truly miss the hallway and empty room meetings 
and quick zoom sessions to not substitute.  And I have had only one 
effective gather session.

> than onsite at this point

NO argument there.  Perhaps better use of jabber by physical attendees?

> - remote is really bad for the informal discussions (gather is very 

No debate.  And really gather was empty, given all who were remote. 
Granted I was only on during 'lunch' which was my breakfast.

> The most efficient WG were where most of the contributurs had decided 

All that I attended (other than plenary) had very empty rooms.  Good 
progress was still made.

> Maybe the IETF needs to rehink the purpouse of onsite meetings.

I am worried about shmoo pushing 1 F2F per year.

>
> We have made the remote experience so good that the "professionalized" 
> aspect of churning out RFCs doesn't really need onsite.
>

---

From: Wes Hardaker <[email-masked]>
Date: Fri, 25 Mar 2022 07:17:19 -0700

Hi Leif,

Thanks for the comprehensive review.  I had some good and bad experiences
both, but in general think it worked "better than I was expecting it to".

One question, in order to figure out what a concrete suggestion I could

> Maybe the IETF needs to rehink the purpouse of onsite meetings.

So in situations where most of the attendees can't attend in person due to
global issues out of our control, are you suggesting we don't hold hybrid
meetings?  What do you think would be a good ratio of in-person to remote
attendance that should be the barrier for deciding whether or not to hold
an on-site meeting?  [we've had remote participation for a long time of
course, but the ratio was far lower]

[note: I have no decision making ability in this area -- I'm just curious]
-- 
Wes Hardaker
USC/ISI

---

From: Leif Johansson <[email-masked]>
Date: Fri, 25 Mar 2022 15:23:09 +0100

> NO argument there.  Perhaps better use of jabber by physical attendees?

I don't think this is a problem with a technical solution.

	Cheers Leif

---

From: Jan-Frederik Rieckers <[email-masked]>
Date: Fri, 25 Mar 2022 15:31:55 +0100

Hi,

as a first time on-site attendee I'd like to add my 2ct here.

I have attended 3 IETFs before, but all of them only remotely. Since my 
interest was in only one working group, I have attended only the meeting 
of this particular WG and (almost) nothing else.

Although the area of interest has not changed for me, I attended a 
session in almost any slot and while I could not really contribute 
there, I have seen some interesting drafts, that I will definitely look 
into after this meeting.

Being on site instead of online definitely has pushed me to actually 
attend these sessions. If I hat participated at home I'd probably have 
only attended one or two sessions of my personal interest.

The second thing as new attendee is the mentioned hallway discussions. I 
have met a number of people here, who I would have never met if I 
attended online.
I don't have the feeling that there was a lack of on site participants 
to create a 'critical mass', but I haven't been to any other IETF 
meeting, so my sample size is 1.

When being on site, opening the gather is not really possible, and for 
the remote participants, you may try to get other work done instead of 
joining the gather.
Also, the psychological barrier to "bust in" or to actively come up to a 
person in gather is quite high for me, so being on site and just 
greeting people on the hallway and then having them invite me into their 
communication has helped me to get to know people.

I don't have a solution for the gap between onsite and gather or the 
gather participation either (maybe a few ideas but probably they only 
sound good in my head).

I don't think that the hybrid meeting was is "the worst of both worlds" 
although it definitely had potential for improvement.

But what my main take-away from this IETF was, and what I would probably 
not have learned if the IETF was completely remote:

* There are a lot of interesting people here and I want to get to know 
them. And knowing them helps you to find out who to ask.
* There are a lot of interesting topics and it is really good to look 
beyond the horizon

As Leif already wrote, the missing human interaction will probably lead 
to a distance between the participants, that could lead to a serious 
turn in the consensus principle and the discussions. If you know the 
person on a personal level and you had a drink with them, your 
discussions will more likely stay on the professional level, if this 
personal touch is missing, the discussions can get personal very quickly 
(I have experienced exactly this behavior in some other contexts already)

 From my experience the IETF is a very welcoming community and I am so 
glad that I had the opportunity to be here in person.

Greetings
Janfred
-- 
E-Mail: [email-masked] | Fon: +49 30884299-339 | Fax: +49 30884299-370
Pronomen: er/sein | Pronouns: he/him
__________________________________________________________________________________

DFN - Deutsches Forschungsnetz | German National Research and Education 
Network
Verein zur Förderung eines Deutschen Forschungsnetzes e.V.
Alexanderplatz 1 | 10178 Berlin
www.dfn.de

Vorstand: Prof. Dr. Odej Kao (Vorsitzender) | Dr. Rainer Bockholt | 
Christian Zens
Geschäftsführung: Dr. Christian Grimm | Jochem Pattloch
VR AG Charlottenburg 7729B | USt.-ID. DE 1366/23822

---

From: Leif Johansson <[email-masked]>
Date: Fri, 25 Mar 2022 15:36:29 +0100

On 2022-03-25 15:17, Wes Hardaker wrote:
> Hi Leif,
> 
> Thanks for the comprehensive review.  I had some good and bad experiences both, but in general think it worked "better than I was expecting it to".
> 
> One question, in order to figure out what a concrete suggestion I could take away from your notes:
> 
> 
> So in situations where most of the attendees can't attend in person due to global issues out of our control, are you suggesting we don't hold hybrid meetings?  What do you think would be a good ratio 
> of in-person to remote attendance that should be the barrier for deciding whether or not to hold an on-site meeting?  [we've had remote participation for a long time of course, but the ratio was far 
> lower]
> 

I'm suggesting that there might come a time (soon probably) where there are no obstacles to travel
but the technology is "good enough" for remote *execpt* for all the reasons the IETF is more than
a set of WGs...

At that point we should either give up on f2f entirely (and I think give up on the IETF as something
more than the I-Ds it produces) *OR* figure out another reason for people to want to travel.

	Cheers Leif

---

From: Rifaat Shekh-Yusef <[email-masked]>
Date: Fri, 25 Mar 2022 16:03:36 +0100

Leif,

I had a very positive experience with the OAuth WG meetings.

We had two official meetings and two side meetings.
With the official meetings, most of the attendees were local, but we had
good participation from remote attendees too.
The side meeting allowed us to discuss other topics that resulted in making
progress that was later presented during the second official meeting,
Another topic was discussed and we have a plan on how to proceed from here.
One random hallway chat with someone that noticed that I chair the OAuth WG
and provided me with verbal feedback on his experience using OAuth. We
discussed that briefly and I asked him if he is willing to share his
thoughts with the WG. As a result, that person created a few slides that
captured his feedback and presented these to the WG the next day during one
of the side meetings.

It would be really difficult to get even closer to what we achieved during
this week if it was completely virtual.

Regards,
 Rifaat

On Fri, Mar 25, 2022 at 3:37 PM Leif Johansson <[email-masked]> wrote:

> > Thanks for the comprehensive review.  I had some good and bad
> experiences both, but in general think it worked "better than I was
> expecting it to".
> >
> > So in situations where most of the attendees can't attend in person due
> to global issues out of our control, are you suggesting we don't hold
> hybrid meetings?  What do you think would be a good ratio
> > of in-person to remote attendance that should be the barrier for
> deciding whether or not to hold an on-site meeting?  [we've had remote
> participation for a long time of course, but the ratio was far
> > lower]
> >
> 113attendees mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/113attendees
>

---

From: Göran Selander <[email-masked]>
Date: Fri, 25 Mar 2022 15:08:54 +0000

>
> From: 113attendees <[email-masked]> on behalf of Leif Johansson <[email-masked]>
>
My company has recently set a permanent cap on travel emissions to 50% of pre-pandemic (2019) levels - the bulk of which will come from air travel. So travel will remain an obstacle as least for some of us.
Göran

---

From: Henk Birkholz <[email-masked]>
Date: Fri, 25 Mar 2022 16:10:50 +0100

Hi Leif,

I am having a really hard time agreeing with your assessment. I was/am 
an onsite attendee. In the WGs I am involved in, this f2f meeting made 
excellent progress, key individuals where one part onsite and two parts 
remote (very subjective, unfounded, non-statistical feeling). And from 
my individual onsite attendee perspective, this was a great success.

I agree that the gather.twon experience was rather limited. Bob actually 
was in my ear one day suddenly (I was muted and gather was always on, 
but I forgot about that). Alas, I was just about to change locations, 
felt bad, and then forgot to come back to him (sorry, Bob!). That makes 
me feel that the "hallway track" experience between onsiters and 
remoters was probably very different.

That said, the "hallway track" feeling of "online IETFs" simply cannot 
be compared to the "f2f experience" it simply is not nearly as good. I 
am aware that a lot of effort went into designing the gather.town sites, 
the scavenger hunts, the "local" scenery, etc. But my side-meetings 
during online meetings where 90% in our "everyday online tools" and I 
did probably attend only 50% of the WGs that I would normally have (also 
due to dayjob and that virtual business trips are a fantasy of mine that 
do not really work out)

I got so much done this IETF onsite! It was amazing! In hindsight that 
is such a stark contrast to the visible, tangible, and really worrying 
decline in sparking new work that I experienced during the online IETF 
meetings. I was so happy to see that onsite... that problem simply went 
away! I would like to quote another (this time still remote) fellow 
IETF'ler here:

> I recall the feeling of strange euphoria that comes from the mix of exhaustion and hyper-communication

I experienced that feeling again, this week. And it is hard to put in 
words how much I was missing that as a human being. I know - looking at 
the numbers - this will not get easy soon. But I am absolutely certain 
about a few things:

* we need these f2f meetings and cannot give up on them
* the next IETF meetings will still be a challenge and the challenges 
will change as will the world around us - all the time
* the onsite crew that made this hybrid meeting possible met an 
impossible challenge with extraordinary creativity and patience and did 
an amazing job!

There is always room for improvement, but I cannot see an IETF with less 
then 3 meetings per year in the long term.

As Jan-Frederick pointed out... especially newcomers will have a really 
hard time integrating into our community - as we need them to! I'll 
accept the obvious fact that some meetings will again be remote in my 
life-time. But I definitely expect that to be the exception to the rule.

Viele Grüße,

Henk

On 25.03.22 15:36, Leif Johansson wrote:
>> So in situations where most of the attendees can't attend in person 
>> due to global issues out of our control, are you suggesting we don't 
>> hold hybrid meetings?  What do you think would be a good ratio of 
>> in-person to remote attendance that should be the barrier for deciding 
>> whether or not to hold an on-site meeting?  [we've had remote 
>> participation for a long time of course, but the ratio was far lower]
>>

---

From: Joris Baum <[email-masked]>
Date: Fri, 25 Mar 2022 16:44:36 +0100

Hi,

I was in the same situation. This was my first on-site IETF and I took 
part in the IETF since 109. You summed it up quite nicely!

I just had a weird idea I wanted to share. Even though the remote 
hallway discussion experience will probably always be a bit lacking 
behind, maybe one way to improve this is to have something like "gather 
areas" on-site. Something like a bridge between virtual and on-site. An 
open space in the hallway with a mic and a big screen where people from 
on-site and gather could talk to each other without a big communication 
barrier.

Regards,

Joris

On 25.03.22 15:31, Jan-Frederik Rieckers wrote:
> I have attended 3 IETFs before, but all of them only remotely. Since 
> my interest was in only one working group, I have attended only the 
> meeting of this particular WG and (almost) nothing else.
>
> there, I have seen some interesting drafts, that I will definitely 
> look into after this meeting.
>
> The second thing as new attendee is the mentioned hallway discussions. 
> I have met a number of people here, who I would have never met if I 
> attended online.
> Also, the psychological barrier to "bust in" or to actively come up to 
> a person in gather is quite high for me, so being on site and just 
> greeting people on the hallway and then having them invite me into 
> their communication has helped me to get to know people.
>
> I don't think that the hybrid meeting was is "the worst of both 
> worlds" although it definitely had potential for improvement.
>
> But what my main take-away from this IETF was, and what I would 
> probably not have learned if the IETF was completely remote:
>
> As Leif already wrote, the missing human interaction will probably 
> lead to a distance between the participants, that could lead to a 
> serious turn in the consensus principle and the discussions. If you 
> know the person on a personal level and you had a drink with them, 
> your discussions will more likely stay on the professional level, if 
> this personal touch is missing, the discussions can get personal very 
> quickly (I have experienced exactly this behavior in some other 
> contexts already)
>

-- 
Joris Baum
Tel: +49 721 170293 16
Fax: +49 721 170293 179

http://www.audriga.com | http://www.twitter.com/audriga

--------------------------------------------------------------------------
audriga GmbH | Alter Schlachthof 57 | 76137 Karlsruhe
Sitz der Gesellschaft: Karlsruhe - Amtsgericht Mannheim - HRB 713034
Geschäftsführer: Dr. Frank Dengler, Dr.-Ing. Hans-Jörg Happel
--------------------------------------------------------------------------

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 25 Mar 2022 16:51:45 +0100

On 2022-03-25, at 16:44, Joris Baum <[email-masked]> wrote:
> 
> I just had a weird idea I wanted to share. Even though the remote hallway discussion experience will probably always be a bit lacking behind, maybe one way to improve this is to have something like "gather areas" on-site. Something like a bridge between virtual and on-site. An open space in the hallway with a mic and a big screen where people from on-site and gather could talk to each other without a big communication barrier.

That idea is not so weird at all.

I didn’t really have time to use gather this time either, but the hjkl interface didn’t really help me find the people that I wanted to “run into”.  Some more sensible people met down at the area tables.  Maybe we can start with those, mirroring them to the physical world.

(The basic idea is very old and can be extended to “half-rooms” for physical-only meetings that occur at different places synchronously.  In most one-room physical meetings with added virtual attendance, you find people sitting beside the camera and talking to some people who are remote, without any benefit of formal meeting arrangements.
We could expand on that.)

Oh, and what Henk said.

I was remote, but definitely benefitted from the additional energy that the on-site participants had.  We built our own half-rooms for some side discussions; that should be supported some more.

Grüße, Carsten

---

From: Antoine FRESSANCOURT <[email-masked]>
Date: Fri, 25 Mar 2022 15:57:52 +0000

Hello,

This was my first in person IETF meeting and while I enjoyed meeting people in person in the hallway and put a face on names I see on the mailing lists, I think the « official agenda » part of the meeting would have been a better experience online than onsite:

- the chat is not available on the mobile site for meetecho and lots of discussions occur there. « Oh it is discussed in the chat » is frustrating when you are in the room.

- often we couldn’t see the face of people talking remotely

- in many WG meetings, I had the impression to attend reporting that could have been an email while actual technical discussions on items of the WG were pushed to the mailing list for the sake of a lack of time.

The last point might be the one that puzzles me the most. What is the point meeting in person to push discussions on online tools ? And why do we need to have WG activity reporting as slides rather than an email ?

My 2 cents,

________________________________

Antoine FRESSANCOURT
Email: [email-masked]<mailto:[email-masked]>

From: Leif Johansson<[email-masked]<mailto:[email-masked]>>
To: 113attendees<[email-masked]<mailto:[email-masked]>>
Subject: [113attendees] hybrid meetings: the worst of both worlds
Time: 2022-03-25 14:36:43

        Cheers Leif

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 12:05:42 -0400

The hackathon room may work for this.  The online participants can go to 
the appropriate table.  I have used the virtual whiteboard; works well 
with my Wacom tablet.

But then there would need to be LCSD screens at the physical tables.

On 3/25/22 11:44, Joris Baum wrote:
> behind, maybe one way to improve this is to have something like 
> "gather areas" on-site. Something like a bridge between virtual and 
> on-site. An open space in the hallway with a mic and a big screen 
> where people from on-site and gather could talk to each other without 
> a big communication barrier.
>
> Regards,
>
> Joris
>
>
>> attend these sessions. If I hat participated at home I'd probably 
>> have only attended one or two sessions of my personal interest.
>>
>> The second thing as new attendee is the mentioned hallway 
>> discussions. I have met a number of people here, who I would have 
>> never met if I attended online.
>> I don't have the feeling that there was a lack of on site 
>> participants to create a 'critical mass', but I haven't been to any 
>> other IETF meeting, so my sample size is 1.
>>
>> When being on site, opening the gather is not really possible, and 
>> for the remote participants, you may try to get other work done 
>> instead of joining the gather.
>> Also, the psychological barrier to "bust in" or to actively come up 
>> to a person in gather is quite high for me, so being on site and just 
>> * There are a lot of interesting people here and I want to get to 
>> know them. And knowing them helps you to find out who to ask.

-- 
Standard Robert Moskowitz
Owner
HTT Consulting
C:248-219-2059
F:248-968-2824
E:[email-masked]

There's no limit to what can be accomplished if it doesn't matter who 
gets the credit

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 12:09:06 -0400

> - the chat is not available on the mobile site for meetecho and lots of discussions occur there. « Oh it is discussed in the chat » is frustrating when you are in the room.

Before all of this many of us used a jabber client to be talking during 
the session.  You can always use whatever system you have with a jabber 
client/server and go to jabber.ietf.org.

And I used a separate jabber client (pidgin), so I was one of many that 
were logged in twice.

> ------------------------------------------------------------------------
>
> Antoine FRESSANCOURT
> Email: [email-masked]
>
> *From: *Leif Johansson<[email-masked]>
> *To: *113attendees<[email-masked]>
> *Subject: *[113attendees] hybrid meetings: the worst of both worlds
> *Time: *2022-03-25 14:36:43
>
>

---

From: NA-NiC <[email-masked]>
Date: Fri, 25 Mar 2022 17:28:07 +0100

Antoine,

I came there to listen, and sat in a number of different tracks in addition to DNSOP which is where my interest lies, and agree with every point you made.

Many presentations were incredibly detailed, with very busy and sometime impossible to read slides, quite long with very little time for debate. While not everybody has good verbal presentation skills, maybe this is something to look at. Kawasaki’s 10-20-30(20) rule comes to mind.

And almost on every single presentation there was an issue with screen sharing in the beginning, which decreased in amusement value quickly over the days.

The chair microphones were in most rooms not loud enough to understand at the back of the rooms, the remote audio was great.

I did like the IETFer App, but would have liked to be able to check in to the Blue Sheet from the App  instead of just the QR.

el

—
Sent from Dr Lisse’s iPhone/iPad
> The last point might be the one that puzzles me the most. What is the point meeting in person to push discussions on online tools ? And why do we need to have WG activity reporting as slides rather…

 […]

---

From: Ted Lemon <[email-masked]>
Date: Fri, 25 Mar 2022 17:36:12 +0100

My experience of the meeting was that I had several really important
conversations that I hadn't planned to have, nor would have had if we
hadn't met. So it was definitely a net positive for me. The hallway
conversations are precisely the thing that we miss being remote. I like the
"gather area" idea, but I think it probably needs a bit more thought, and
we'd have to arrange it so that people who didn't want to be on camera
weren't accidentally captured, while still making it something you'd run
into randomly. A difficult conundrum.

As to the actual WG meetings, I am realizing that one of the main purposes
of these meetings is to trigger hallway discussions. I don't think that was
ever as obvious to me as it's been this time, because before this it was
just what happened normally. IOW, not that the sessions don't matter at
all—they do—but the synergy of sessions triggering discussions is what
makes in-person IETF meetings what they are.

Leif, I suspect that I was luckier than you were about which people with
whom I needed to talk to showed up in person. Of course having most people
show up in person makes that less of a crap shoot.

On Fri, Mar 25, 2022 at 5:09 PM Robert Moskowitz <[email-masked]>
wrote:

> ------------------------------
>
> Antoine FRESSANCOURT
> Robert Moskowitz
> C:      248-219-2059
> F:      248-968-2824
> E:      [email-masked]
>
> There's no limit to what can be accomplished if it doesn't matter who gets
> the credit
> --

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 12:38:31 -0400

On 3/25/22 12:28, NA-NiC wrote:
> Antoine,
>
> I came there to listen, and sat in a number of different tracks in 
> addition to DNSOP which is where my interest lies, and agree with 
> every point you made.
>
> Many presentations were incredibly detailed, with very busy and 
> sometime impossible to read slides, quite long with very little time 
> for debate. While not everybody has good verbal presentation skills, 
> maybe this is something to look at. Kawasaki’s 10-20-30(20) rule comes 
> to mind.

I had always tried to download the slides and follow along when I was in 
the room unless I was sitting right up front.  Of course that was before 
my cataract surgery (just before this nonsense) so I can probably now 
handle all the way back to the middle of the room!  :)

And jabber is great.  You can be sitting in one session and jabbering 
about another.

> And almost on every single presentation there was an issue with screen 
> sharing in the beginning, which decreased in amusement value quickly 
> over the days.
>
> The chair microphones were in most rooms not loud enough to understand 
> at the back of the rooms, the remote audio was great.

This was a real issue for us remote.  Often even with the sound all the 
way up, I could not hear the person(s) at the desk.  then a remote 
speaker would come in and I have to quickly turn the volume down.  Part 
of the problem was mask wearing; that tended to muffle speakers.

>
> I did like the IETFer App, but would have liked to be able to check in 
> to the Blue Sheet from the App  instead of just the QR.
>
> el
>
> —
> On 25. Mar 2022, 16:58 +0100, Antoine FRESSANCOURT 
> <antoine.fressancourt=[email-masked]>, wrote:
>
>     Hello,

---

From: Behcet Sarikaya <[email-masked]>
Date: Fri, 25 Mar 2022 11:39:25 -0500

I also think that the current hybrid format is very good and will likely
continue for a long time to come.

Onsite attendance is good if you can and probably you get the maximum out
of your attendance. Remote is also good even though ideal.
I was remote and because of the time difference and the scheduling of
sessions over which I have no control I had to miss several sessions I
hoped to attend
(e.g. I think all BOFs were scheduled 4-6am my time). But I can check the
minutes and read the documents, etc. to catch up.

The attendance was low expectedly, actually I thought it would be even
lower so maybe that's why Leif couldn't get what he expected.

My suggestion is to continue like this, what's wrong with it?

Behcet

On Fri, Mar 25, 2022 at 10:04 AM Rifaat Shekh-Yusef <[email-masked]>
wrote:

> The side meeting allowed us to discuss other topics that resulted in
> making progress that was later presented during the second official
> meeting, Another topic was discussed and we have a plan on how to proceed
> from here.
> One random hallway chat with someone that noticed that I chair the OAuth
> WG and provided me with verbal feedback on his experience using OAuth. We

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 12:44:32 -0400

On 3/25/22 12:39, Behcet Sarikaya wrote:
> I also think that the current hybrid format is very good and will 
> likely continue for a long time to come.
>
> Onsite attendance is good if you can and probably you get the maximum 
> out of your attendance. Remote is also good even though ideal.
> (e.g. I think all BOFs were scheduled 4-6am my time). But I can check 
> the minutes and read the documents, etc. to catch up.
>

An alternative would be to have THREE physical sites.  This would cut 
down on CO2 from travel (meet shmoo goal) and get more into the venues.  
Big challenge to have multiple multi-site setups.  There are 
professional services that offer small such meeting sites; can that 
model be scaled up to what we would need?

Would really stretch support services and probably raise the attendance 
cost.

>
> Behcet
>
> On Fri, Mar 25, 2022 at 10:04 AM Rifaat Shekh-Yusef 
> <[email-masked]> wrote:
>
>     Leif,
>
>     With the official meetings, most of the attendees were local, but
>     we had good participation from remote attendees too.
>     The side meeting allowed us to discuss other topics that resulted
>     in making progress that was later presented during the second
>     official meeting, Another topic was discussed and we have a plan
>     on how to proceed from here.
>     One random hallway chat with someone that noticed that I chair the
>     OAuth WG and provided me with verbal feedback on his experience
>     using OAuth. We discussed that briefly and I asked him if he is
>     willing to share his thoughts with the WG. As a result, that
>     person created a few slides that captured his feedback and
>     presented these to the WG the next day during one of the side
>     meetings.
>
>     It would be really difficult to get even closer to what we
>     achieved during this week if it was completely virtual.
>
>     Regards,
>      Rifaat
>
>
>         > Thanks for the comprehensive review.  I had some good and
>         bad experiences both, but in general think it worked "better
>         than I was expecting it to".
>         >
>         > One question, in order to figure out what a concrete
>         suggestion I could take away from your notes:
>         >
>         >
>         > So in situations where most of the attendees can't attend in
>         person due to global issues out of our control, are you
>         suggesting we don't hold hybrid meetings?  What do you think
>         would be a good ratio
>         > of in-person to remote attendance that should be the barrier
>         for deciding whether or not to hold an on-site meeting? 
>         [we've had remote participation for a long time of course, but
>         the ratio was far
>         > lower]
>         >

---

From: Ted Lemon <[email-masked]>
Date: Fri, 25 Mar 2022 17:56:11 +0100

Several of the people I was happy to run into at this IETF are in
Asia/Pacific locations, so I suspect this would be a lot of cost to do
something that materially makes things worse, unfortunately.

Personally I think we should just have IETF in a giant zeppelin that starts
in one continent, picks people up as it follows the trade winds, and
ultimately drops us all off about two weeks after we boarded, well
socialized and happy. But I'm sure people will claim that I'm a dreamer
who's just floating a trial balloon that will never go anywhere.

On Fri, Mar 25, 2022 at 5:45 PM Robert Moskowitz <[email-masked]>
wrote:

> An alternative would be to have THREE physical sites.  This would cut down
> on CO2 from travel (meet shmoo goal) and get more into the venues.  Big
> challenge to have multiple multi-site setups.  There are professional
> services that offer small such meeting sites; can that model be scaled up
> to what we would need?
>
> On Fri, Mar 25, 2022 at 10:04 AM Rifaat Shekh-Yusef <
> [email-masked]> wrote:
>
>> Leif,
>>
>> It would be really difficult to get even closer to what we achieved
>> during this week if it was completely virtual.
>>
>> Regards,
>>  Rifaat
>>
>>
>>> due to global issues out of our control, are you suggesting we don't hold

---

From: Jen Linkova <[email-masked]>
Date: Fri, 25 Mar 2022 18:58:59 +0100

> more than the I-Ds it produces) *OR* figure out another reason for people to want to travel.

As my experience this week was completely different, I - rather
selfishly, indeed - hope that your proposal wouldn't be implemented
any time soon.
In the last 5 days I managed to make more progress than in the last 2
years and I can't wait for Philadelphia already.
IMHO IETF f2f meetings are a great example of holism, when the whole
is greater than the sum of its parts.
It's not about presenting a draft during a WG session or having a
meeting/drink with colleagues. It's also about having the latter right
after the former - while energy and enthusiasm are still high. The
sessions and presentations give us food for thoughts and discussions
between sessions (so IMHO we shall have much longer breaks).

For me there is another benefit of f2f meetings, even if not everyone
is here. There are WGs which I'm actively participating in (well, at
least I try..) - during the virtual meetings I was joining those
sessions even if it was 2am my time. But there are many other sessions
which I found very educational, and I did learn a lot this week by
attending them. It's a completely different story for virtual IETFs
though -  I just can't afford attending everything I'm curious about..

To sum it up: I wish there were more people, but still this IETF is
the best one for me since 2020.

--
SY, Jen Linkova aka Furry

---

From: Mike Bishop <[email-masked]>
Date: Fri, 25 Mar 2022 18:03:22 +0000

Indeed. The number of passionate disagreements that can be solved by "The four people who care are going to go have lunch, then present a consensus position at tomorrow's session" cannot be understated. In virtual land, the arguments stretch on for weeks or months.

Sent from Nine<http://www.9folders.com/>
________________________________
From: Jen Linkova <[email-masked]>
Sent: Friday, March 25, 2022 1:59 PM
To: Leif Johansson
Cc: [email-masked]; Wes Hardaker
Subject: Re: [113attendees] hybrid meetings: the worst of both worlds

---

From: <[email-masked]>
Date: Fri, 25 Mar 2022 11:07:17 -0700

Extremely well said.

From: 113attendees <[email-masked]> On Behalf Of Mike Bishop
Sent: Friday, March 25, 2022 11:03 AM
To: Jen Linkova <[email-masked]>; Leif Johansson <[email-masked]>
Cc: [email-masked]; Wes Hardaker <[email-masked]>
Subject: Re: [113attendees] hybrid meetings: the worst of both worlds

Indeed. The number of passionate disagreements that can be solved by "The
four people who care are going to go have lunch, then present a consensus
position at tomorrow's session" cannot be understated. In virtual land, the
arguments stretch on for weeks or months.

Sent from  <http://www.9folders.com/> Nine

  _____  

From: Jen Linkova <[email-masked] <mailto:[email-masked]> >
Sent: Friday, March 25, 2022 1:59 PM
To: Leif Johansson
Cc: [email-masked] <mailto:[email-masked]> ; Wes Hardaker
Subject: Re: [113attendees] hybrid meetings: the worst of both worlds

-- 
113attendees mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/113attendees

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 14:20:05 -0400

Like how we set about resolving the different positions espoused in rfcs 
1597 and 1627...

Ah, life, and the IETF, seemed so simple back then...

On 3/25/22 14:03, Mike Bishop wrote:
> Indeed. The number of passionate disagreements that can be solved by 
> "The four people who care are going to go have lunch, then present a 
> consensus position at tomorrow's session" cannot be understated. In 
> virtual land, the arguments stretch on for weeks or months.
>
> Sent from Nine <http://www.9folders.com/>
> *From:* Jen Linkova <[email-masked]>
> *Sent:* Friday, March 25, 2022 1:59 PM
> *To:* Leif Johansson
> *Cc:* [email-masked]; Wes Hardaker
> *Subject:* Re: [113attendees] hybrid meetings: the worst of both worlds
>

---

From: Azael Fernandez Alcantara <[email-masked]>
Date: Fri, 25 Mar 2022 12:36:30 -0600

Hi all,

We have to face the new reality we are living and are going to live 
because of more pandemics (we are still in one), climate
change and due to more conflicts.

I think hybrid meetings have to remain, and try to find better ways to 
resolve the issues mentioned to keep going with the discussions and 
interactions.

Only face to face meetings, could be part of past memories.

BEST,
SALUDOS,
_______
Azael

On Fri, 25 Mar 2022, Robert Moskowitz wrote:

> Like how we set about resolving the different positions espoused in rfcs 1597 and 1627...
> 
>       Indeed. The number of passionate disagreements that can be solved by "The four people who care are going to
>       go have lunch, then present a consensus position at tomorrow's session" cannot be understated. In virtual
>       land, the arguments stretch on for weeks or months.
> 
> Sent from Nine
> 
> _________________________________________________________________________________________________________________________
> There's no limit to what can be accomplished if it doesn't matter who gets the credit
> 
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 25 Mar 2022 14:42:17 -0400

Also, three physical sites would mean that I would not have hallway 
conversations ad hoc with folks at other physical sites.  A rather 
important problem if we want to keep this as one IETF.

Yours,
Joel

---

From: "John Levine" <[email-masked]>
Date: 25 Mar 2022 15:12:29 -0400

>important problem if we want to keep this as one IETF.

If each site needed enough meeting rooms for parallel tracks, a
network, and cookie breaks, it'd roughly triple our costs so I think
this has to be a complete non-starter.  Yes, smaller rooms and
fewer cookies are cheaper but that much cheaper.

R's,
John

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 16:21:43 -0400

It is just a pipe dream from disturbed sleeping patterns (meetings at 
odd hours).  :)

Perhaps that airship idea is not so full of hot air!

On 3/25/22 15:12, John Levine wrote:
> It appears that Joel M. Halpern<[email-masked]>  said:

---

From: Michael Richardson <[email-masked]>
Date: Fri, 25 Mar 2022 21:30:11 +0100

> NO argument there.  Perhaps better use of jabber by physical attendees?

I think that the same number of people were directly in jabber as usual.
Yes, the huge influx of meetecho mediated people were not there because the
on-site meetecho did not do chat.  Some people did use the full interface
on-site, but for me, the "killer" reason to use the on-site version from the
mobile phone is that it gave me the slide clicker.

    > No debate.  And really gather was empty, given all who were
    > remote. Granted I was only on during 'lunch' which was my breakfast.

On a laptop screen there just isn't space to do gather.
On a dual screen desktop, it works.  But more importantly, during the breaks,
which is when people might to do gather, we, on-site, were collecting cookies
and chatting.

If gather had a place where I could walk up to a big screen (with my kaffe
und kekse) and chat with remote people, I would have done that.

    > All that I attended (other than plenary) had very empty rooms.  Good

The rooms were in general very empty.
I took pictures, partly to reduce anxiety of my family.

    > I am worried about shmoo pushing 1 F2F per year.

There are pluses and minues about such a process, and I think that we might
get to such a thing in ten years.   I think that probably we should.
But, I don't know how to get there, and I don't think we should force.
(I also don't think that shmoo is pushing towards that *AT ALL*)

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Ira McDonald <[email-masked]>
Date: Fri, 25 Mar 2022 16:50:30 -0400

Fascinating thread and diverse viewpoints - thanks - I love the global
airship...

My two cents - for those of us who are independents without any hope of
travel
budget from clients, international travel for ten days to a pricey hotel is
*way* out
of the question.  I've been joining IETF conferences remotely for more than
20
years and will continue to do so.

Cheers,
- Ira

On Fri, Mar 25, 2022 at 4:30 PM Michael Richardson <[email-masked]>
wrote:

>     >> - remote-participation is arguably better for the technical WG
> process
>     >> than onsite at this point
>
>     >> The most efficient WG were where most of the contributurs had
> decided
>     >> to show up onsite.
>

---

From: Bob Hinden <[email-masked]>
Date: Fri, 25 Mar 2022 14:07:55 -0700

I don’t think the hybrid meeting was the worst of both worlds, that would have been no meeting at all.

I didn’t travel to this meeting for several reasons, so instead I got to join in three 2am sessions.   I can't say it was fun, but it did work.   Had I been on site, I would have attended more sessions and had lots of side meetings, but that doesn’t work well given the time the meeting was for me (2am to 8:30am).

I hope to travel to future meetings, but at least for now I think it’s going to be a while before we get back to the attendance that the IETF used to have onsite.

I thought the hybrid meeting worked well.   I actually liked the single queue for local and remote participants.  I think it worked pretty well (except for some people having trouble getting into the queue with the mobile meetecho app).   I suspect this will get better next time.

Hopefully see you at IETF 114.

Bob

---

From: Michael Richardson <[email-masked]>
Date: Fri, 25 Mar 2022 22:11:58 +0100

Carsten Bormann <[email-masked]> wrote:
    > I was remote, but definitely benefitted from the additional energy that
    > the on-site participants had.  We built our own half-rooms for some
    > side discussions; that should be supported some more.

A number of rooms (IESG, HOST) had monster TVs with hard to understand
interfaces that we tried to use.  For HOMENET rechartering Eric tried to make
the TV work with webex, but audio didn't pass through, so we gave that up.

A lot of these conference systems support SIP.  At least Zoom and webex have
SIP URLs that lead into them.  It's kinda amazing that the conference screen
may actually drag us out of the vendor proprietary vertical back into
standards based communication.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Michael Richardson <[email-masked]>
Date: Fri, 25 Mar 2022 22:14:42 +0100

Antoine FRESSANCOURT <antoine.fressancourt=[email-masked]> wrote:
    > - in many WG meetings, I had the impression to attend reporting that
    > could have been an email while actual technical discussions on items of
    > the WG were pushed to the mailing list for the sake of a lack of time.

    > The last point might be the one that puzzles me the most. What is the
    > point meeting in person to push discussions on online tools ?

This is an astute and intelligent question.
Many have been asking this question for ten years.
Maybe it's time to hire a professional of some kind to answer this question.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 25 Mar 2022 22:16:01 +0100

Ted Lemon <[email-masked]> wrote:
    > As to the actual WG meetings, I am realizing that one of the main
    > purposes of these meetings is to trigger hallway discussions. I don't
    > think that was ever as obvious to me as it's been this time, because
    > before this it was just what happened normally. IOW, not that the
    > sessions don't matter at all—they do—but the synergy of sessions
    > triggering discussions is what makes in-person IETF meetings what they
    > are.

The meeting content and time are forcing functions to get everyone into the
same context.   That's some combination of time, place and topic.

---

From: Michael Richardson <[email-masked]>
Date: Fri, 25 Mar 2022 22:18:14 +0100

el> And almost on every single presentation there was an issue with screen
    el> sharing in the beginning, which decreased in amusement value quickly
    el> over the days.
    el>
    el> The chair microphones were in most rooms not loud enough to understand
    el> at the back of the rooms, the remote audio was great.

"Doctor, it hurts when I sit at the back of the room"
-> don't sit at the back of the room.

    > of the problem was mask wearing; that tended to muffle speakers.

I would like us to just put fewer chairs in the room, and have tables for
everyone.

---

From: David Waite <[email-masked]>
Date: Fri, 25 Mar 2022 15:27:07 -0600

As a remote participant, I definitely felt like I had a second class technical and social experience. This translated into radically less participation. Unfortunately, this is a very common pattern for hybrid events and not something I would not expect to be fixed without quite a bit of additional, ongoing efforts. 

- in one session, my audio when speaking came through so quiet that it was unusable. Humorously, there was also quite a loud delayed echo of my own voice as I spoke, significantly louder than in-person participants were getting. AFAICT, there was no attempt to resolve this technical issue, which occurred multiple times before my attempt.

- in another session, the audio was miscalibrated badly enough for the first 5-10 minutes that I turned audio off for the presentation until there was an indication in chat that it was fixed. If the audio in-person was that poor, I doubt the session would have started.

- in yet another session, a remote presenter found themselves unable to do so due to technical issues. My impressions were these technical issues were not due to their local environment or networking. Luckily, there was someone onsite as a backup for the presentation.

- Meetecho had unusable overlapping layout on mobile, at least as initiated from the calendar events. The session I attempted to take via my iPhone did not appear to have the presentation visible or available due to these problems.

- Meetecho seemed to respond to any network or technical glitch in the individual streams would cut audio and video off until the page was reloaded, without indication of error. This made me especially dislike how there is a multi-step camera-and-mic setup on each reload. I had to do this multiple times in each of the sessions I attended.

- the majority of side sessions were not remotely accessible

- hallway conversations don’t have a viable mechanism yet to pull the two groups of participants together.

From this experience, I have no intention to participate in future IETF unless I can either participate onsite or it is a fully remote event.

I regret attempting to attend this IETF. With the issues I had participating in discussions as well as viewing live presentations, I feel I would have made far more efficient use of my time just reviewing slides and minutes.

-DW

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 17:32:36 -0400

> I would like us to just put fewer chairs in the room, and have tables for

For hybrid this makes sense.  But maybe not if Philly gets 1,000 
personal attendance.

And those rooms at the Metropole may not be good for more tables. ;)

---

From: NA-NiC <[email-masked]>
Date: Fri, 25 Mar 2022 22:55:48 +0100

It didn’t hurt as much as reading some of your posts, it was inaudible and I am quite doubtful that it was a function of distance to the loudspeakers, even though my ENT rotation is 40 years behind me.

Remote participants have told us for them it was even worse.

el

—

---

From: Leif Johansson <[email-masked]>
Date: Fri, 25 Mar 2022 23:13:31 +0100

its ironic that this point appears to have been somewhat lost in the thread: I do *not* believe that remote can replace onsite, I just believe we will be defeated by laziness if we’re not careful…. Remote is comfortable, just not good for us.

Cheers Leif

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 25 Mar 2022 18:20:41 -0400

> its ironic that this point appears to have been somewhat lost in the thread: I do *not* believe that remote can replace onsite, I just believe we will be defeated by laziness if we’re not careful….…

I think there is general agreement here that remote can replace onsite.

And it is not always comfortable to be remote.  Weird hours and all 
that.  And home obligations being at hand...

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 26 Mar 2022 00:23:56 +0100

Hmm..

> Indeed. The number of passionate disagreements that can be solved by "The four people who care are going to go have lunch, then present a consensus position at tomorrow's session" cannot be underst…

If exactly and only the very same four people would have had lunch together via 
gather.town in a privat corner, how much different/worse would the result have been ?

IMHO, if those people knew each other from in before in person, the virtual
meeting would come quite close, maybe even better if some more tooling at home/gather
would have helped (restaurant napkins have their limits).

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Sat, 26 Mar 2022 01:29:47 +0100

Ok. I'll bite. Here is my hybrid rant:

One wish i have is that people on-site should be remote-inclusive whenever
they do announce/organize an activity that easily could include remote
participation.

This week i had to again to an announcement of an ad-hoc side-meeting at a WG
meeting, where immediately remote participants asked if/how to participate remotely,
and the organizer said "no remote participation, but we'll send notes". IMHO that
is unacceptable. Especially when it is planned to be meeting somewhere in the
IETF hotel, where it is easily possible to just have a notebook or cell phone with
gather.town or any RTCweb tool and a Jabra Speak for great audio with remote
participants.

Maybe a few Jabra Speak loaner at the registration desk or the like could help.
and some easy wiki page explaining easy setups.  Its really puzzling how we're
inventing so much great dog food for remote, and then at the easiest of opportunities,
we don't eat it.

And for those remote-seggregationists who explicitly want to discriminate,
i would suggest that announcements about any such activities can
ONLY go to an on-site "white"-board, like what we used in the ietf before we
started to use these remote-friendly tools like wiki pages. But IMHO, whenever
remote accessible IETF/WG space is used to announce activities, we should look for
those to be remote-friendly as much as possible.

And no: using the words "bar" and "bof" together is not a valid excuse for
remote discrimination. Especially given how every time the audio of a chosen
location is too bad to include remote participants for an actual technical discussion, 
it is likely also too bad to include non-native-english participants well (SnR/comprehension/...).

At least i had exactly that frustrating experience in several actual loud-bar BoF
meetings in the past, which i totally despised for exactly that reason. But i think
there are now fewer of those.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Yoav Nir <[email-masked]>
Date: Sat, 26 Mar 2022 07:07:32 +0300

Hi, Rifaat

> On 25 Mar 2022, at 18:03, Rifaat Shekh-Yusef <[email-masked]> wrote:
> 
> Leif,
> 
> With the official meetings, most of the attendees were local, but we had good participation from remote attendees too.
> The side meeting allowed us to discuss other topics that resulted in making progress that was later presented during the second official meeting, Another topic was discussed and we have a plan on how to proceed from here.
> One random hallway chat with someone that noticed that I chair the OAuth WG and provided me with verbal feedback on his experience using OAuth. We discussed that briefly and I asked him if he is willing to share his thoughts with the WG. As a result, that person created a few slides that captured his feedback and presented these to the WG the next day during one of the side meetings.
> 
> It would be really difficult to get even closer to what we achieved during this week if it was completely virtual.

But compare and contrast that with the experience of I2NSF, where there were 3-6 people in the room (other than you), and where the chairs, the presenters, pretty much everyone but Diego were remote.

Was the I2NSF experience any better than a virtual interim?

Yoav

BTW: Thanks again for sitting in for us.

---

From: NA-NiC <[email-masked]>
Date: Sat, 26 Mar 2022 06:42:26 +0100

I did not hear any humming.

Hybrid is the way to go, in my view, as it does allow for all the things possible when onsite, including to see the Phillies play the Cubs or Braves.

On the other hand people from developing countries, which are woefully underrepresented through no fault of the IETF, mind you, can participate easily, as can all who can’t (afford to) travel.

greetings, el

—

---

From: Dr Eberhard W Lisse <[email-masked]>
Date: Sat, 26 Mar 2022 06:53:07 +0100

What irritated me a bit, were several speakers who did not queue on the meetecho, and just mumbled their name, so even sitting there I had no idea who was speaking.

I think it would be a great idea to provide some form of infrastructure to assist in breaking out.

el

—

---

From: Rifaat Shekh-Yusef <[email-masked]>
Date: Sat, 26 Mar 2022 09:11:04 +0100

Hi Yoav,

That meeting was bad, but would moving it to fully virtual meeting have
made it any better? I do not think so, since, as you already mentioned, the
majority were remote anyway.

There are so many factors that can impact the performance of a WG: the
number of people attending, the interests of people in the work being
discussed, the energy in the WG, etc.

I guess I am arguing against fully virtual meeting in the future. There is
no way that we can get even close to the results of meeting in person.

Regards,
 Rifaat

On Saturday, March 26, 2022, Yoav Nir <[email-masked]> wrote:

> But compare and contrast that with the experience of I2NSF, where there
> were 3-6 people in the room (other than you), and where the chairs, the
> presenters, pretty much everyone but Diego were remote.
>

---

From: <[email-masked]>
Date: Sat, 26 Mar 2022 08:39:13 +0000

Just my 2 cents,

while I agree to all the details, I don’t subscribe to the summary “worst of both worlds”.
Meetecho did a good job to integrate on-site.
And I DID experience fruitful hallway discussions, even though not as much as in earlier times.
And I think that will get better with every percent we can increase on-site participation.

Given the current situation I believe hybrid meetings are the right thing to do, and I hope that on-site will increase over time.

Maybe we can way ways to foster on-site participation a bit.
(As for my special case, public transport’s “Fahrgeldnachforderung” didn’t help. Prague had a much nicer solution.)

Best regards,
Martin

Von: 113attendees <[email-masked]> im Auftrag von Leif Johansson <[email-masked]>
Datum: Freitag, 25. März 2022 um 14:35
An: [email-masked] <[email-masked]>
Betreff: [113attendees] hybrid meetings: the worst of both worlds

        Cheers Leif

---

From: Ted Lemon <[email-masked]>
Date: Sat, 26 Mar 2022 11:09:17 +0100

> Perhaps that airship idea is not so full of hot air!

The idea comes from one of Kim Stanley Robinson’s books, probably
Antarctica. Highly recommend. I don’t expect to see anything like this in
my lifetime, but I’d love to be proven wrong. :)

---

From: Ted Lemon <[email-masked]>
Date: Sat, 26 Mar 2022 11:20:16 +0100

> gather.town in a privat corner, how much different/worse would the result

In my case at least it simply wouldn’t have happened, since I’ve never
found gather usable, and so wouldn’t have been at this putative gathering.
It would be better to meet via zoom. But that’s the thing: definitely
meetings happened on person that have not been happening for me at remote
IETFs.

Maybe this is because I’m an aged dinosaur, but I suspect it’s more than
that. Better tools might improve this situation, of course. But not gather.

---

From: Yoav Nir <[email-masked]>
Date: Sat, 26 Mar 2022 14:00:48 +0300

A virtual interim would be a little better. It would be at a time convenient to all of us, we would have just the interested people (even if that’s 5) and we can have them whenever it’s convenient, not just in IETF week.

Avoiding that picture of a big room with three people sitting in the back is also a plus.

As we learn how to make these hybrid meetings work, we may want to avoid having groups like that meet within IETF week. Virtual interims are still available. Perhaps it’s better to leave that slot available to groups that can make good use of it, like OAuth or TLS or LAMPS (to name just a few groups I know that could use more time).

Yoav

> On 26 Mar 2022, at 11:11, Rifaat Shekh-Yusef <[email-masked]> wrote:
> 
> Hi Yoav,
> 
> That meeting was bad, but would moving it to fully virtual meeting have made it any better? I do not think so, since, as you already mentioned, the majority were remote anyway. 
> 
> There are so many factors that can impact the performance of a WG: the number of people attending, the interests of people in the work being discussed, the energy in the WG, etc. 
> 
> I guess I am arguing against fully virtual meeting in the future. There is no way that we can get even close to the results of meeting in person. 
> 
> Regards,
>  Rifaat
> 
> 
> On Saturday, March 26, 2022, Yoav Nir <[email-masked] <mailto:[email-masked]>> wrote:
> Hi, Rifaat
> 
> > On 25 Mar 2022, at 18:03, Rifaat Shekh-Yusef <[email-masked] <mailto:[email-masked]>> wrote:
> > 
> > Leif,
> >

---

From: Michael Richardson <[email-masked]>
Date: Sat, 26 Mar 2022 13:54:07 +0100

I've been doing 1 meeting remote since around 2004.
I got multicast IP to work once, but then the IETF abandonned that method.
I know all about how remote people get "excluded", but ...

Toerless Eckert <[email-masked]> wrote:
    > One wish i have is that people on-site should be remote-inclusive
    > whenever they do announce/organize an activity that easily could
    > include remote participation.

I don't have that wish.
I wish that people who want to organize side meetings with remote people
would just have a "zoom" meeting next week, with all the people remote.

    > And no: using the words "bar" and "bof" together is not a valid excuse

I want us to return to bar bofs happenning in bars with beer and napkins.

---

From: "Livingood, Jason" <[email-masked]>
Date: Sat, 26 Mar 2022 14:20:13 +0000

This was the IETF’s first COVID-era* hybrid meeting. I wish I could have been there in person, but it didn't work out. I imagine that - as usual - there will be some lessons learned that will be applied next time at IETF-114. This hybrid meeting seemed to go generally well, amidst great uncertainty and an extremely dynamic global health environment. I really appreciate everyone's efforts to carry off the meeting so successfully given all of that!!

The next meeting will be in Philadelphia in their (our) summertime, which is typically a time when respiratory viruses are at a low point. It is also usually quite warm & humid and people can be comfortable outdoors**. So I am hopeful we may see more in-person attendance, by who knows***.

Jason

* I’m trying to train myself to say COVID-era rather than post-COVID. ;-)
** There are several small outdoor parks within 1 - 4 blocks walk of the venue. The city has the Fairmont Park system, which is 4,100 acres in size
*** I wonder if it is worth opening the 114attendees list early so people can discuss things related to attendance before the start of registration (which is usually the trigger for creating the list)? It might cut down on the noise on other lists.

---

From: Ted Lemon <[email-masked]>
Date: Sat, 26 Mar 2022 15:23:47 +0100

Starting the list early sounds like a good plan.

On Sat, Mar 26, 2022 at 10:20 Livingood, Jason <Jason_Livingood=
[email-masked]> wrote:

> This was the IETF’s first COVID-era* hybrid meeting. I wish I could have
> been there in person, but it didn't work out. I imagine that - as usual -
> there will be some lessons learned that will be applied next time at
> IETF-114. This hybrid meeting seemed to go generally well, amidst great
> uncertainty and an extremely dynamic global health environment. I really
> appreciate everyone's efforts to carry off the meeting so successfully
> given all of that!!
>
> The next meeting will be in Philadelphia in their (our) summertime, which
> is typically a time when respiratory viruses are at a low point. It is also
> usually quite warm & humid and people can be comfortable outdoors**. So I
> am hopeful we may see more in-person attendance, by who knows***.
>
> Jason
>
>
> ** There are several small outdoor parks within 1 - 4 blocks walk of the
> venue. The city has the Fairmont Park system, which is 4,100 acres in size
> *** I wonder if it is worth opening the 114attendees list early so people
> can discuss things related to attendance before the start of registration
> (which is usually the trigger for creating the list)? It might cut down on
> the noise on other lists.
>
> --

---

From: Leif Johansson <[email-masked]>
Date: Sun, 27 Mar 2022 00:33:51 +0100

> I want us to return to bar bofs happenning in bars with beer and napkins.

Yep

---

From: Kyle Rose <[email-masked]>
Date: Sat, 26 Mar 2022 19:04:56 -0600

> I want us to return to bar bofs happenning in bars with beer and napkins.

Agreed.

More generally, there would be little point in traveling were all
interactions handicapped to the degree that would be required for full
remote participant equity. The IETF should provide reasonable accommodation
for remote attendees (which IMO includes things like the new local/remote
mic queue discipline) but accept that there is value in *being there* that
cannot be replicated by remote attendees via any near-term telepresence
technology.

Moreover, the IETF should not apologize for wanting to maximize the
benefits of the reality we inhabit. These are constraints not under our
control.

Kyle

---

From: Dan Romascanu <[email-masked]>
Date: Sun, 27 Mar 2022 04:39:05 +0300

Hi Antoine,

This is because it may be that the point of view of the reporting persons (typically the chairs) may be different than the one of other people in the room (local or remote). So having these reported on slides together with other meeting materials helps preparing for such an occasional debate, which may be important, and archiving for later scrutiny.

Regards,

Dan

Sent from my iPhone

> On 25 Mar 2022, at 18:58, Antoine FRESSANCOURT <antoine.fressancourt=[email-masked]> wrote:
> 
>  And why do we need to have WG activity reporting as slides rather than an email ?

---

From: Rifaat Shekh-Yusef <[email-masked]>
Date: Sun, 27 Mar 2022 06:43:09 -0400

On Sat, Mar 26, 2022 at 7:00 AM Yoav Nir <[email-masked]> wrote:

> A virtual interim would be a little better. It would be at a time
> convenient to all of us, we would have just the interested people (even if
> that’s 5) and we can have them whenever it’s convenient, not just in IETF
> week.
>
> As we learn how to make these hybrid meetings work, we may want to avoid
> having groups like that meet within IETF week. Virtual interims are still
> available. Perhaps it’s better to leave that slot available to groups that
> can make good use of it, like OAuth or TLS or LAMPS (to name just a few
> groups I know that could use more time).
>

This is a really good point, Yoav.
The OAuth WG did not meet during the Virtual IETF meeting period.
Instead, usually after each Virtual IETF meeting, we scheduled a *series of
1-hour interim meetings*, each one dedicated to discuss *one document* in
detail.
This could be a model that such groups could use, to avoid taking time
slots needed by other WGs that could use more time.

Regards,
 Rifaat

>> > With the official meetings, most of the attendees were local, but we
>> had good participation from remote attendees too.
>> OAuth WG and provided me with verbal feedback on his experience using
>> OAuth. We discussed that briefly and I asked him if he is willing to share
>> his thoughts with the WG. As a result, that person created a few slides
>> that captured his feedback and presented these to the WG the next day
>> during one of the side meetings.
>> >

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 27 Mar 2022 21:39:45 +0100

Hiya,

On 25/03/2022 22:13, Leif Johansson wrote:
> its ironic that this point appears to have been somewhat lost in the
> thread: I do *not* believe that remote can replace onsite, I just
> believe we will be defeated by laziness if we’re not careful…. Remote
> is comfortable, just not good for us.

I agree. I thought the meeting went very well, given all the
factors involved. (So I don't agree with Leif's choice of
subject line.)

I would though encourage the IESG to be more radical with
the f2f agenda time than was the case for 113. My particular
suggestion/hobby-horse would be to run the hackathon Mon-Thu
in the mornings and WG sessions/plenary/other things in the
afternoons. As others have said, we could also encourage use
of online interims for WGs and topics that don't really need
f2f time, so I'd also bump up the occurrence of "other
things" in those afternoon sessions, e.g. perhaps a session
devoted to revisiting the overall documentation of DNS as has
been suggested on dnsop. Regardless of whether those specific
suggestions are considered, I do think we need to move away
from the kind of agenda structure we used prior to 2020.

Anyway, it was really great to see a good few people this
week. And I look forward to seeing more who couldn't make it
this time, and to meeting new people, at IETFs to come.

Cheers,
S.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 10:03:28 +0200

These mail threads of course encourage primarily contrarian pondering.
I think we do get more substantial understanding of the overall community
opinion by those questionaires that JohnD started to do. Aka: it would be
useful to have over time more question with the experience with the different
type of get-togethers. As long as they cover relevnt questions. Such as 
usefulness or its opposite in the face of different audio qualities of a
particular venue (such as a bar).

Wrt to resisting more remote-inclusiveness: Its clear that if we do not have
a group trying to push the envelope, we wouldn't have moved away from the
local-only state we had 30 years ago. That's just human nature. And i am sure
that if we would look back, we'll find enough evidence of participants rejecting
even the idea of what we have achieved so far (with meetecho/gather) as impossible
or undesirable back then.

I have just seen outside of IETF over the decades a lot more remote-inclusive
options that i think are also worth to explore for IETF. Sitting in a side-meeting
room in the IETF and NOT having a remote participation option for example strikes
me as particularily objectionable. Sitting in a bar and trying to get actual
technical work done, but then not understanding anyone due to background noise
equally so.

Yes, spending more time by doing local and then remote meetings would be nice,
but i already know that only a minority of people that where able to meet locally
would make themselves available remotely to the same degree.

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Jay Daley <[email-masked]>
Date: Mon, 28 Mar 2022 09:27:58 +0100

Hi Toerless

> On 28/03/2022, at 9:03 AM, Toerless Eckert <[email-masked]> wrote:
> 

Side meetings are deliberately very different from other sessions as repeatedly stated by the IESG:

https://www.ietf.org/blog/public-side-meetings/

Significant effort has been put into improving the remote experience during IETF meetings but, for the reasons in that blog post, not side meetings.  While I understand your frustration regarding side meetings, an assessment of remote participation for the whole meeting based on an experience in a side meeting will never be fair and representative. 

Jay

-- 
Jay Daley
IETF Executive Director 
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 10:49:39 +0200

> Significant effort has been put into improving the remote experience during IETF meetings but, for the reasons in that blog post, not side meetings.  While I understand your frustration regarding s…

Thanks, Jay

Wrt to side meetings, there are two different frustrations:

Deliberately providing physical space (rooms), but not webex space for side
meeting strikes me as highly illogical. That is an IESG policy issue. I claim it
is a remote-unfriendly policy.

Deliberately NOT using remote participation tools in a side-meeting strikes
me as highly remote-unfriendly. That is a side-meeting organizer issue.

The linkage is of course that side meeting organizers not well versed in remote-friendly
tooling might have an easier time if the IETF policy would make its tools available
to them. Like it is for finding a room.

Aka: as long as IETF policy in support of remote participation in side meetings
is as it is compared to its support for in-person meeting (rooms), its really
difficult to give side-meeting organizers a hard time when they choose NOT to support
remote participation. 

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Lars Eggert <[email-masked]>
Date: Mon, 28 Mar 2022 12:38:20 +0300

Hi,

> is a remote-unfriendly policy.

the IETF is not "deliberately providing physical space", i.e., we do not contract extra meeting space so that side meetings can be held.

What we do offer is the use of rooms that are used for other purposes some of the time (e.g., for IESG and IAB breakfast meetings, etc.) when they would otherwise sit empty.

We provide a wiki for people to coordinate the use of these rooms during such times, so that they can avoid scheduling during times when the rooms are blocked for their original purposes, and they can also avoid conflicts with each other. (Which did happen before the wiki was offered for coordination.)

> me as highly remote-unfriendly. That is a side-meeting organizer issue.

I agree.

> difficult to give side-meeting organizers a hard time when they choose NOT to support

Most participants by now have two year's experience with running online meetings as part of their dayjobs, and most have access to their organizational Zoom/Webex/etc. setups. I remain unconvinced that a lack of knowledge or access to an A/V service is the issue here.

Thanks,
Lars

---

From: Benoit Claise <[email-masked]>
Date: Mon, 28 Mar 2022 11:51:49 +0200

Dear all,

> factors involved. (So I don't agree with Leif's choice of
+1

Even we had a limited amount of people on side, I had a couple of very 
interesting meetings that made it worth the trip.
What I have witnessed with previous online meetings is that each author 
cares mainly about his draft: basically presenting and pushing his 
"stuff", without that much interaction about other draft presentations. 
I also believe (no numbers to support this though) that, during this 
period, less new drafts were submitted.
With this first hybrid meeting, I found back the hackathon/hallway/bar 
discussions, the ones that generate new innovations: new research, 
future draft, and potential hackathon ideas. Granted, it really depends 
if your "keys" people are physically present. However, having less 
people on site helped me to spend more quality time with each person, 
instead of rushing it.

Regards, Benoit

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 28 Mar 2022 12:57:22 +0300

Toerless Eckert writes:
> > Indeed. The number of passionate disagreements that can be solved
> > by "The four people who care are going to go have lunch, then
> > present a consensus position at tomorrow's session" cannot be
> > understated. In virtual land, the arguments stretch on for weeks
> > or months.
> 
> If exactly and only the very same four people would have had lunch
> together via gather.town in a privat corner, how much
> different/worse would the result have been ?

The problem is that is it is almost impossible to get it working in
gather or other online tool. Taking your four person exmple. Firstly
one of the four people would not be in the meeting at all, as it
happened to be 3am for him, and he did not consider the meeting
important enough to wake just for that. For onsite meeting it would be
normal working hours thus he would wonder in to meeting that is not
really important to him, but which he knows lot about, meaning his
input to the discussion would be very important.

Secondly one of those four people would have company meeting right
after the session, thus they would not be able to attend this kind of
lunch meeting at that short notice. Third person would most likely
want to fix some lunch at home during the break, and could not
concentrate on the lunch meeting while actually preparing food
(compared to the case where restaurant staff makes food for you). So
only one of the four people would really end up in gather, and then he
would wonder what was there that was supposed to be discussed here...

> IMHO, if those people knew each other from in before in person, the
> virtual meeting would come quite close, maybe even better if some
> more tooling at home/gather would have helped (restaurant napkins
> have their limits).

When working out issues, I think any webex/zoom/jitsi/meetecho meeting
is not as good as having people in the same meeting room and actually
concentrating on the meeting. With all these tooling people are very
often not fully concentrating themselves to the meeting, as it is so
easy to do other things at the same time and other people in the
meting does not usually notice it. I myself do it all the time during
virtual meetings. On the other hand I sometimes do it also during
onsite meetings, but then that is usually much more noticeable to
others in the meeting, and they will wake me up if there is something
important to discuss...

During this IETF there were several cases where I was able to solve an
issue, just by seeing the right person during the break and talking to
him. If I would have needed to go and hunt him through gather, or
arrange special zoom etc meetign with him that would have never
happened, as I only really realized that he is correct person to ask
when I actually saw him on break.
-- 
[email-masked]

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 28 Mar 2022 13:43:27 +0300

Toerless Eckert writes:
> This week i had to again to an announcement of an ad-hoc
> side-meeting at a WG meeting, where immediately remote participants
> asked if/how to participate remotely, and the organizer said "no
> remote participation, but we'll send notes". IMHO that is
> unacceptable. Especially when it is planned to be meeting somewhere
> in the IETF hotel, where it is easily possible to just have a
> notebook or cell phone with gather.town or any RTCweb tool and a
> Jabra Speak for great audio with remote participants.

It is not just that simple. It took us about quite some time (more
than hour I think) during the codesprint to get the room projector to
work and actually project the video streams of offsite participants,
and then we ended up having issues with sound, as laptop speakers were
not loud enough to be usable in big room. They fixed this by attaching
someones bluetooth speaker to the laptop, but even then there was
issues with microphones, and if you wanted talk to someone remotely,
you needed to move next to laptop connected to the projector to be
able to use its microphone.

During the one of the ADs office hours, the only way we could get
connection from remote ADs to room, was to use someones laptop and its
microphone and speakers. That worked as there was only one or two
people talking to them at the same time.

The amount of work done to get the audio and video working properly
during the WG sessions is quite a lot and is not something that can be
done on 5 minutes notice on random meeting room.

It is completely different to have virtual meeting where everybody is
using their own laptops to connect, than to have virtual meeting where
there is big room with 10 people connected to the virtual meeting.

> Maybe a few Jabra Speak loaner at the registration desk or the like
> could help. and some easy wiki page explaining easy setups. Its
> really puzzling how we're inventing so much great dog food for
> remote, and then at the easiest of opportunities, we don't eat it.

We are using our own dog food, but we did not make audio/video
equipments in IETF, thus that is not our dog food... 

> And for those remote-seggregationists who explicitly want to
> discriminate, i would suggest that announcements about any such
> activities can ONLY go to an on-site "white"-board, like what we
> used in the ietf before we started to use these remote-friendly
> tools like wiki pages. But IMHO, whenever remote accessible IETF/WG
> space is used to announce activities, we should look for those to be
> remote-friendly as much as possible.

I do not think there is people who want to actively discriminate
against anybody, but setting things up is not just that simple, and
usually takes time, and people are much more keen on using the time to
actually do work, than to set up audio and video systems for half an
hour or so.
-- 
[email-masked]

---

From: Tero Kivinen <[email-masked]>
Date: Mon, 28 Mar 2022 14:05:04 +0300

> Avoiding that picture of a big room with three people sitting in the back is

When you have two other security area working groups meeting at the
same time than yours, then there is less participation...

> available. Perhaps it’s better to leave that slot available to groups that can
> make good use of it, like OAuth or TLS or LAMPS (to name just a few groups I
> know that could use more time).

I was considering to come to the i2nsf, but then decided to go to suit
instead. I would never consider joing to i2nsf virtual meeting, as
I am not that interested in i2nsf even when I have participated it
onsite few times before...

So the question is do you think this kind of random walkups are usuful
for the WG or not. I think they are useful and thats why I think
actual onsite meetings are better than virtual interims.
-- 
[email-masked]

---

From: "Dean Bogdanovic" <[email-masked]>
Date: Mon, 28 Mar 2022 07:45:04 -0400

On 28 Mar 2022, at 5:51, Benoit Claise wrote:

> What I have witnessed with previous online meetings is that each 
> author cares mainly about his draft: basically presenting and pushing 
> his "stuff", without that much interaction about other draft 
> presentations. I also believe (no numbers to support this though) 
> that, during this period, less new drafts were submitted.
> future draft, and potential hackathon ideas. Granted, it really 
> depends if your "keys" people are physically present. However, having 
> less people on site helped me to spend more quality time with each 
> person, instead of rushing it

+1
IMO, the hybrid meeting was a success. The first day, there were some 
technical hiccups, but we got very fast around it. The WG chairs (if not 
present), found an in room coordinator that was able to bridge virtual 
and physical presence if needed. The side meetings and conversations 
were great and it was good to see many people in person on site, not 
just for work, but also socially. Because of work I started to follow 
new WG and having ability to ask few questions to get quick 
clarifications, is invaluable.

Dean

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 28 Mar 2022 05:23:01 -0700

>
>
>
> Taking your four person exmple. Firstly
[...]

Good analogies in that group of examples (all true).  But the flip side is
also true though: person 5 in this scenario was left out of the 4-count
entirely because they would have never interacted with the IETF since it
was too expensive for his company/world-region to consider attending and
thus their voice and perspective was lost.  I'm sure the IETF has lost some
participants in the last 2 years because of the lack of in-person
engagement, but we've also picked up a few that only participated for the
first time because it was possible for the first time [I had a number of
good conversations with a few of them in gather].

> meting does not usually notice it.

FYI, my perception is this is about an equal split.  My pictures of rooms
full of IETF members before and after laptops became the norm are very
different.  Afterward, 90% of the audience is looking at their laptops.  It
would be interesting to figure out a way to see if people notice (eg) slide
changes more or less frequently in person for in-virtual, since if you have
a monitor devoted to the slides you might see the slide change and wake
your attention a bit.  It seems that way to me but I have no way to measure
it.

Don't get me wrong: face-to-face is better in so so many ways.  But trying
to boost the tools for remote participation can only help make
participation levels itself better, even if we get back to the point where
the larger percentage is in person.

-- 
Wes Hardaker
USC/ISI

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 28 Mar 2022 08:47:04 -0400

My concern is that the way understand Toerless' characterization and 
requirement would prevent many useful itneractions.

the canonical case in my view is that a bunch of people are having a 
discussion in the meeting.  They decide to continue the discussion 
afterwards.  As corutesy to the meeting they arrange to use one of the 
available rooms, and they let the WG know where and when they will 
continue talking so that those who want to can come.

Toerless seems to declare that if the participants can not practically 
arrange remote participation options for that discussion then tehy can 
not have that meeting as described?   That seems like hurting ourselves.

It should be remembered that wide ranging discussions can be difficult 
even face to face.  Providing full effective remote participation is 
desirable, but is hard and in my view can't be a requirement for 
permitting such side meetings.  (The practical reduction of effective 
discussion in WG meetings is much smaller and clearly acceptable.  We 
have already, for good reasons, made sure that the room can hear such 
things.)

Yours,
Joel

On 3/28/2022 8:23 AM, Wes Hardaker wrote:
> Good analogies in that group of examples (all true).  But the flip side 
> is also true though: person 5 in this scenario was left out of the 
> 4-count entirely because they would have never interacted with the IETF 
> since it was too expensive for his company/world-region to consider 
> attending and thus their voice and perspective was lost.  I'm sure the 
> IETF has lost some participants in the last 2 years because of the lack 
> of in-person engagement, but we've also picked up a few that only 
> participated for the first time because it was possible for the first 
> time [I had a number of good conversations with a few of them in gather].
> 
> FYI, my perception is this is about an equal split.  My pictures of 
> rooms full of IETF members before and after laptops became the norm are 
> very different.  Afterward, 90% of the audience is looking at their 
> laptops.  It would be interesting to figure out a way to see if people 
> notice (eg) slide changes more or less frequently in person for 
> in-virtual, since if you have a monitor devoted to the slides you might 
> see the slide change and wake your attention a bit.  It seems that way 
> to me but I have no way to measure it.
> Don't get me wrong: face-to-face is better in so so many ways.  But 
> trying to boost the tools for remote participation can only help make 
> participation levels itself better, even if we get back to the point 
> where the larger percentage is in person.
> 
> -- 
> Wes Hardaker
> USC/ISI
>

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 28 Mar 2022 08:53:22 -0400

On 3/26/22 01:53, Dr Eberhard W Lisse wrote:
> What irritated me a bit, were several speakers who did not queue on 
> the meetecho, and just mumbled their name, so even sitting there I had 
> no idea who was speaking.
>
> I think it would be a great idea to provide some form of 
> infrastructure to assist in breaking out.

Enhancing the hackathon room/tables might be a good approach.  It would 
help remote hackathon participants as well as hybrid discussions.

>> whenever
>> could help.
>> discriminate,
>>> purposes
>>>> This was my first in person IETF meeting and while I enjoyed 
>>>> meeting people in person in the hallway and put a face on names I 
>>>> see on the mailing lists, I think the « official agenda » part of 
>>>> the meeting would have been a better experience online than onsite:
>>>>
>>>>
>>>> - the chat is not available on the mobile site for meetecho and 
>>>> lots of discussions occur there. « Oh it is discussed in the chat » 
>>>> is frustrating when you are in the room.
>>>>
>>>>
>>>> - in many WG meetings, I had the impression to attend reporting 
>>>> that could have been an email while actual technical discussions on 
>>>> items of the WG were pushed to the mailing list for the sake of a 
>>>> lack of time.
>>>>
>>>>
>>>> The last point might be the one that puzzles me the most. What is 
>>>> the point meeting in person to push discussions on online tools ? 
>>>> decided to
>>>> show up onsite.

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 28 Mar 2022 09:01:40 -0400

Yes we can discuss what we found challenging and people can discuss if 
there are ways to work on those challenges.

On 3/26/22 10:23, Ted Lemon wrote:
> On Sat, Mar 26, 2022 at 10:20 Livingood, Jason 
> <Jason_Livingood=[email-masked]> wrote:
>
>     This was the IETF’s first COVID-era* hybrid meeting. I wish I
>     could have been there in person, but it didn't work out. I imagine
>     that - as usual - there will be some lessons learned that will be
>     applied next time at IETF-114. This hybrid meeting seemed to go
>     generally well, amidst great uncertainty and an extremely dynamic
>     global health environment. I really appreciate everyone's efforts
>     to carry off the meeting so successfully given all of that!!
>
>     The next meeting will be in Philadelphia in their (our)
>     summertime, which is typically a time when respiratory viruses are
>     at a low point. It is also usually quite warm & humid and people
>     can be comfortable outdoors**. So I am hopeful we may see more
>     in-person attendance, by who knows***.
>
>     Jason
>
>
>     ** There are several small outdoor parks within 1 - 4 blocks walk
>     of the venue. The city has the Fairmont Park system, which is
>     4,100 acres in size
>     *** I wonder if it is worth opening the 114attendees list early so
>     people can discuss things related to attendance before the start
>     of registration (which is usually the trigger for creating the
>     list)? It might cut down on the noise on other lists.
>
>     --

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 28 Mar 2022 09:04:40 -0400

>> I want us to return to bar bofs happenning in bars with beer and napkins.

Virtual napkins can work as well.  If we can have portable virtual 
flipcharts that someone physically at the bar can run, then those remote 
can add their sketches to the 'napkins'.

Plus then you have them and should not have to worry about waitstaff 
taking them when your attention is elsewhere!

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 16:00:29 +0200

Tero,

I think it must have been something like 10 years of online meeting while working
inside a company that was actually selling the meeting tool, before every required
participant and the tool itself was 97% able to avoid getting stuck with
wasting 10 minutes in the beginning of each meeting with meeting logistics.

Nothing comes for free. We all had to learn how to read and write, and then
how to do it appropriately for email, mailing lists, across various type
of computers and so on.

Yes: If someone wants to feel qualified to step in and say "hey, i can help
with that remote experience", then he/she should be able to

a) have some tool understood - webex/zoom/gather or one out of 100 more RTCweb
   conference tools (whereby seems nice too).
b) Know how to connect a projector to an HDMI output of his notebook,
   and maybe have the cable.
c) Have something like a blueooth speaker for a room and give instructions
   to local participants how to use it.
d) Have remote participants that can be persuaded to wear headsets when they
   cause echo (i failed with my colleague this time... twice).

I am sure qhite a few IETF participants are happy to help with this type of
setup issues.

Its really NOT rocket science. By now its a very simple craft everyone can learn.
In comparison, learning english was harder for me, and we expect every IETF participant
to do just that (imagine how great IETF would be if like in ITU we would provide
translation services).

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 16:02:15 +0200

Ted,

i remember your concerns about gather. I wonder if/what steps there might be
for mitigation, e.g.: feedback for improvement or the like.

On your other point: I am sure most of us are dinosaurs, and i always fear the
time when a young person tells me all problems of the world can be solved with
tools not larger than smart-phones, but ideally watches. That is when i am
typically declare defeat on my side.

Cheers
    Toerless

On Sat, Mar 26, 2022 at 11:20:16AM +0100, Ted Lemon wrote:
> > together via

-- 
---
[email-masked]

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 16:13:12 +0200

On Mon, Mar 28, 2022 at 08:47:04AM -0400, Joel M. Halpern wrote:
> arrange remote participation options for that discussion then tehy can not
> have that meeting as described?   That seems like hurting ourselves.

In that particular case i got so annoyed about this time, the person who
promoted the ad-hoc side-meeting on the WG microphone in the room during the
WG meeting could have simply asked the room "if there is anyone who could help doing
a remote participation setup, we would appreciate it". That did not happen.

"Good faith effort to not exclude remote" when you are explicitly talking to
a hybrid audience ? Its perfectly fine for remote to be not included once 
you're after the meeting organizing something in-person only with in-person
people.

> It should be remembered that wide ranging discussions can be difficult even
> face to face.  Providing full effective remote participation is desirable,
> but is hard and in my view can't be a requirement for permitting such side
> meetings.

Sure. Participation selection and management is a multi-faceted problem. But i
think that local/remote is in many cases orthogonal to other considerations or
at least it should be IMHO. And i feel that instead, we see location as a welcome
pseuo-argument for discrimination. If you're remote and you're considered important
by the organizers, they will do anything they can to still make it (meeting)
happen with you. If you're not considered important by the organizers, then
they're happy to use the remote argument for exclusion. And i simply disapprove
of thinking like that (as one example).

Cheers
    Toerless

> (The practical reduction of effective discussion in WG meetings
> is much smaller and clearly acceptable.  We have already, for good reasons,
> made sure that the room can hear such things.)
> 
> Yours,
> Joel
> 

-- 
---
[email-masked]

---

From: Dr Eberhard W Lisse <[email-masked]>
Date: Mon, 28 Mar 2022 16:25:23 +0200

I do not think that the organizers are at fault here, they seem to
have gone out of their way to support both on-site and remote
participation, and am sure that they will provide some form of
infrastructure at future meetings to assist with breaking out
:-)-O

What I observed repeatedly, however, in this context, was
thoughtlessness from participants and occasionally what I (a
newcomer) felt was arrogance.

el

On 2022-03-28 16:13 , Toerless Eckert wrote:
 >> Toerless seems to declare that if the participants can not
 >> practically arrange remote participation options for that
 >> discussion then tehy can not have that meeting as described?
 >> That seems like hurting ourselves.
 >
 > In that particular case i got so annoyed about this time, the
 > person who promoted the ad-hoc side-meeting on the WG microphone
 > in the room during the WG meeting could have simply asked the
 > room "if there is anyone who could help doing a remote
 > participation setup, we would appreciate it".  That did not
 > happen.
 >
 > "Good faith effort to not exclude remote" when you are
 > explicitly talking to a hybrid audience ?  Its perfectly fine
 > for remote to be not included once you're after the meeting
 > organizing something in-person only with in-person people.
 >
 >> It should be remembered that wide ranging discussions can be
 >> difficult even face to face.  Providing full effective remote
 >> participation is desirable, but is hard and in my view can't be
 >> a requirement for permitting such side meetings.
 >
 > Sure.  Participation selection and management is a multi-faceted
 > problem.  But i think that local/remote is in many cases
 > orthogonal to other considerations or at least it should be
 > IMHO. And i feel that instead, we see location as a welcome
 > pseuo-argument for discrimination.  If you're remote and you're
 > considered important by the organizers, they will do anything
 > they can to still make it (meeting) happen with you.  If you're
 > not considered important by the organizers, then they're happy
 > to use the remote argument for exclusion.  And i simply
 > disapprove of thinking like that (as one example).
 >
 > Cheers
 >      Toerless
[...]

-- 
Dr Eberhard W Lisse
Managing Director
Namibian Network Information Center (Pty) Ltd.
PO Box 8421 Bachbrecht 10007 Namibia
If this email is signed with GPG, Sect 20 Act No 4 of 2019 may apply

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 16:36:27 +0200

> infrastructure at future meetings to assist with breaking out

This discuss was only about side-meetings, not official IETF business,
but of course, there is the official code of conduct for side meetings
from the IETF chair reciting IESG side meeting policy and that code of conduct
does not talk about encouraging enablemenet of remote participation for
side-meetings. And given how we only had remote meetings and this is about the
first time where this was a real problem, its really a new thing. That
type of encouragement would be what IETF officially could at least say
IMHO about side-meetings.

> newcomer) felt was arrogance.

Dismissiveness and other acts of rough conversation are time honored traditions
of IETF participants and leadership, and as a newcomer it is expected of you to 
act assertive and respectful (maybe even obnoxious ;-) against them so communication styles
slowly improve.

In the old days, you could gain easy initial participation credits by yelling
"mike"(rophone) into the room 10 times, when local participants where not
doing it during a meeting, but engage in local conversation. Alas, i think we've
gotten somewhat more advanced now, so you may need to engage in real ugly credits like note
taking or the like - or maybe suggesting to help with remote participation setup ? ;-)

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Benson Muite <[email-masked]>
Date: Mon, 28 Mar 2022 18:04:53 +0300

.
> to do just that (imagine how great IETF would be if like in ITU we would provide
For some languages automated translation and transcription is available. 
This is something that could be investigated.  Correcting the 
transcriptions in addition to or rather than taking notes would provide 
more data for training artificial intelligence models to improve future 
quality.

---

From: Dr Eberhard W Lisse <[email-masked]>
Date: Mon, 28 Mar 2022 17:14:42 +0200

While I am all for making attendance and participation easier
especially from developing countries, English on B2 or higher
level is required to understand any RFC or relevant document, and
hence I do not understand this request.

el

On 2022-03-28 17:04 , Benson Muite wrote:
 > .
 >> In comparison, learning english was harder for me, and we
 >> expect every IETF participant to do just that (imagine how
 >> great IETF would be if like in ITU we would provide translation
 >> services).
 >>
 > For some languages automated translation and transcription is
 > available.  This is something that could be investigated.
 > Correcting the transcriptions in addition to or rather than
 > taking notes would provide more data for training artificial
 > intelligence models to improve future quality.
 >

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 17:15:05 +0200

Yes, google translate, especially the live-camera to screen translation has been
very helpful for me, especially with chinese. And i very much hope that starting
up with IETF for non-native speakers these days is also easier than it might have
been in the past due to such tools. But i think once you start wanting
to contribute to the IETF work product, you will be happy when you have spent
cycles on training your brain itself with english as opposed to training solely
the use of tools to avoid that.

Then again, if we are going down the rabbit hole of ever more computer languages
forming the core of our work product, we may wake up one day and see how the english
in our RFCs has become just irrelevant filler words. And i am not even sure how
much of that evolution i wouldn't call a good thing.

Cheers
    Toerless

On Mon, Mar 28, 2022 at 06:04:53PM +0300, Benson Muite wrote:
> This is something that could be investigated.  Correcting the transcriptions
> in addition to or rather than taking notes would provide more data for
> training artificial intelligence models to improve future quality.
> 
> -- 

-- 
---
[email-masked]

---

From: Michael Douglass <[email-masked]>
Date: Mon, 28 Mar 2022 12:31:31 -0400

https://phys.org/news/2021-12-virtual-conferences-environment-inclusive.html

On 3/28/22 07:45, Dean Bogdanovic wrote:
>> Even we had a limited amount of people on side, I had a couple of 
>> very interesting meetings that made it worth the trip.
>> With this first hybrid meeting, I found back the 
>> hackathon/hallway/bar discussions, the ones that generate new 
>> innovations: new research, future draft, and potential hackathon 
>> ideas. Granted, it really depends if your "keys" people are 
>> physically present. However, having less people on site helped me to 
>> spend more quality time with each person, instead of rushing it
>
> +1
> technical hiccups, but we got very fast around it. The WG chairs (if 
> not present), found an in room coordinator that was able to bridge 
> virtual and physical presence if needed. The side meetings and 
> conversations were great and it was good to see many people in person 
> on site, not just for work, but also socially. Because of work I 
> started to follow new WG and having ability to ask few questions to 
> get quick clarifications, is invaluable.
>
> Dean
>
>>
>> Regards, Benoit
>>
>> --

---

From: Robert Moskowitz <[email-masked]>
Date: Mon, 28 Mar 2022 12:38:23 -0400

It is one thing to be able to read and write English clearly.  It is 
quite another challenge to speak it clearly over remote video, or even 
live sessions.  Especially to those of us with hearing processing 
challenges as well.

I can point to participants in the past from non-english speaking 
countries with superb writing abilities who have taken many of us to 
task about our writing skills with accents so thick as to be a real problem.

---

From: Toerless Eckert <[email-masked]>
Date: Mon, 28 Mar 2022 19:45:09 +0200

Rob: Indeed

When working in the industry, we had public speaking classes, sometimes even
mandated by the organizers (if you want to speak in front of NNNN customers,
you have to qualify). Maybe there would be business that IETF might be happy
to promote for companies educating public speaking to offer such a training
course to IETF'er interested, e.g.: on saturday before IETF or the like.

One of the most simple things to learn (doh!) is to practice your talk so that
you can give it without having to read from your own slides (or a script if
you have good slides with just the pictures). At least until you manage to
read from slides/script without sounding like a robot who was previously
fired as a bad rap singer - e.g.: extreme stakkato, zero sentence structure
or any other structuring intonation - because you're focussed on reading,
not talking.

In research conferences during covid, when everybody had to just submit video
clips of their talks, i did on occasion experience talks where professional
voice actors had been hired by the researchers to dub their video clip.
That was in between "slick" and "foul" to me. But it would be fun to consider hiring a
voice actor for the IETF and letting that actor present slides from a written
script  on behalf of speakers who feel to insecure to present their slides
themselves. At least it would be a fun one-time experiment. Of course only with a posh
british accent! John Cleese would be my favourite.

Cheers
    Toerless

On Mon, Mar 28, 2022 at 12:38:23PM -0400, Robert Moskowitz wrote:
> It is one thing to be able to read and write English clearly.  It is quite
> another challenge to speak it clearly over remote video, or even live
> sessions.  Especially to those of us with hearing processing challenges as
> well.
> 
> I can point to participants in the past from non-english speaking countries
> with superb writing abilities who have taken many of us to task about our
> writing skills with accents so thick as to be a real problem.
> 

-- 
---
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Mar 2022 15:07:01 +0200

Stephen Farrell <[email-masked]> wrote:
    > I would though encourage the IESG to be more radical with the f2f
    > agenda time than was the case for 113. My particular
    > suggestion/hobby-horse would be to run the hackathon Mon-Thu in the
    > mornings and WG sessions/plenary/other things in the afternoons. As
    > others have said, we could also encourage use of online interims for

It's an interesting suggestion, and I think we should try it somehow.
It may be too radical a change for some.

(As a fellow morning-hater, I would personally find that order a problem.
I hack better in the evening, and sessions are a good motivation to wake up)

I wrote that we should have far fewer sessions during f2f time in my
"many-fine-dinners" draft, which was not I think, well received by SHMOO.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 28 Mar 2022 16:40:39 -0400

I continue to hope that one day we will have sessions where we don't get
presentations of the form "here's what changed since last time" that last
ten minutes. But that requires chairs to vet presentations in advance, and
I realize that's hard.

---

From: Behcet Sarikaya <[email-masked]>
Date: Mon, 28 Mar 2022 15:44:20 -0500

> I hack better in the evening, and sessions are a good motivation to wake

Michael, Morning changes based on where you are like for IETF 113, all
sessions happened in the morning for me attending from Dallas :)

Behcet

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 28 Mar 2022 21:52:56 +0100

Hiya,

> It may be too radical a change for some.

Fair point. And it'd have to be done in some way so as to
avoid big new costs (e.g. for re-arranging rooms). OTOH, if
there're only small numbers in the WG sessions maybe keeping
round-table setups would work all week.

Again though, even if that particular suggestion isn't tried,
I suspect being more radical than 113 is going to be needed
if we're gonna end up succeeding with the hybrid meetings
that are inevitable for the coming years. (As someone else
said, climate change alone is likely to be sufficient to
force that.)

> I hack better in the evening, and sessions are a good motivation to wake up)

Wait, what? Isn't the IETF meeting agenda supposed to be
designed around me? :-)

Cheers,
S.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Mar 2022 17:05:06 -0400

Lars Eggert <[email-masked]> wrote:
    >> Deliberately providing physical space (rooms), but not webex space for
    >> side meeting strikes me as highly illogical. That is an IESG policy
    >> issue. I claim it is a remote-unfriendly policy.

    > the IETF is not "deliberately providing physical space", i.e., we do
    > not contract extra meeting space so that side meetings can be held.

    > What we do offer is the use of rooms that are used for other purposes
    > some of the time (e.g., for IESG and IAB breakfast meetings, etc.) when
    > they would otherwise sit empty.

Yes.  I was going to answer Toerless that given his position, maybe
it should be solved by not making the space available. (I'm 6/10 serious)
Force people back to the bar.

Aren't we supposed to have immersive virtual reality by now?
(And flying cars powered by fusion)

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Mar 2022 17:02:20 -0400

Toerless Eckert <[email-masked]> wrote:
    > Sitting in a side-meeting room in the IETF and NOT having a
    > remote participation option for example strikes me as particularily
    > objectionable.
    
    > Sitting in a bar and trying to get actual technical work
    > done, but then not understanding anyone due to background noise equally
    > so.

I think that there is space for a wide set of different options.
I have objected to providing space for side-meetings for many years now.
They become some kind of alternate track and frankly many people are over
scheduled too much already during IETF plenary week.

I agree with you that we have made great strides in supporting online
meetings, even prior to pandemic.

That's why I question the value of side-meetings w/remote participation.
Either:
  1) just schedule a virtual interim meeting or design-team meeting
     if it's associated with a WG already.
  2) or it's really a BOF in disguise.
  3) or it's just a group of IETF people preparing for some BOF.

#3 does not need to happen during IETF plenary week.
Let's just do #1 or #3 at a different time, and all be "remote"

The only *value* that doing it doing IETF plenary week is that is preferences
the time zone in which the meeting is.  When the meeting is in Europe or
America, then that's not really helping spread the pain.  So, it's only
really when the IETF plenary is in Asia does having it during the IETF
plenary week make it less painful for asian participants to join.

    > Yes, spending more time by doing local and then remote meetings would
    > be nice, but i already know that only a minority of people that where
    > able to meet locally would make themselves available remotely to the
    > same degree.

I don't really understand this.

I mean, I understand what you are saying, and I agree that there are people
that one is able to sometimes pin down during IETF plenary week that it can
be hard to pin down outside of that week... but I don't understand why those
people are so important to have at the meeting, given that they aren't
available.

I had two design-team meetings (both involving ADs) last week.

They were mostly arranged last minute, and we did suck in at least one remote
person via a laptop on a table and webex/zoom/..   yes, they were meetings of
opportunity, in order to resolve some issues.  We probably could have
eventually gotten through things via email, or a specific meeting time, but
we didn't.

---

From: David Waite <[email-masked]>
Date: Mon, 28 Mar 2022 16:28:59 -0600

My position is that the idea of being remote-inclusive means that it is not a hybrid event - it is an in-person event with some remote participation options.

In 2020/2021 I attended events which were remote, with an in-person attendance option. There are drastic differences between in-person-first vs remote-first experiences.

For example, a remote-first setup may involve pre-recorded presentations with separate Q&A, split schedules with replayed presentations for chunks of time zones. You also see less parallel, multi-week schedule since travel and venue costs weren’t their typical burdens. The IETF equivalent was a push for established groups to instead do virtual interims.

I have a suspicion that there is no happy hybrid medium between the two. You see some things done entirely different with remote-first events to take advantage of the remote experience. You seem to have a trade-off between dynamic engagement and inclusiveness - but you have network effects at play, so you lose quite a bit when shooting for the middle.

Instead, we should continue to use virtual meetings and in-person IETF meetings as two separate tools within a remotely accessible process.

-DW

> On Mar 25, 2022, at 6:29 PM, Toerless Eckert <[email-masked]> wrote:
>

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Mar 2022 18:36:43 -0400

Ted Lemon <[email-masked]> wrote:
    > I continue to hope that one day we will have sessions where we don't
    > get presentations of the form "here's what changed since last time"

Yes.

I am sometimes harassed by chairs who want to know why I haven't send slides
about what changed since last time.

I almost ALWAYS reply to the ID announcements with exactly that kind of info.

    > that last ten minutes. But that requires chairs to vet presentations in
    > advance, and I realize that's hard.

you forgot the <sarcasm> tag.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 28 Mar 2022 18:43:26 -0400

> Fair point. And it'd have to be done in some way so as to avoid big new
    > costs (e.g. for re-arranging rooms). OTOH, if there're only small
    > numbers in the WG sessions maybe keeping round-table setups would work
    > all week.

Round-table setups could actually be awesome with an agenda of more workshop
and less slideware.

But maybe there is some intermediate arrangement that satisfies both.
Maybe we could have the half-round table where the other half is virtual.

    > Again though, even if that particular suggestion isn't tried, I suspect
    > being more radical than 113 is going to be needed if we're gonna end up
    > succeeding with the hybrid meetings that are inevitable for the coming
    > years. (As someone else said, climate change alone is likely to be
    > sufficient to force that.)

I agree completely.

    >> (As a fellow morning-hater, I would personally find that order a
    >> problem.  I hack better in the evening, and sessions are a good
    >> motivation to wake up)

    > Wait, what? Isn't the IETF meeting agenda supposed to be designed
    > around me? :-)

The trick is for us to always get you to a time zone that is 6 hours ahead of
the meeting "9am"

I find that I can do meetings early (even 3am start) because there are other
people to engage me.  But, I can't write code/edit documents until after
noon, because that kind of activity has to be internally driven.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 28 Mar 2022 18:55:04 -0400

Having been a chair, and having been a presenter, I know that it genuinely
is hard: we don't get to know what will be presented until close to the
meeting, but have to schedule time for the presentation well in advance of
the meeting because that affects how much time we are allocated. I'm not
sure this is a good approach—it might be better to just give each WG that
thinks they need to meet an hour and have unstructured time for discussions
that doesn't come out of the presentation budget. But this is more of a
hand-wave than a serious suggestion—I don't know how to make this work.

---

From: Toerless Eckert <[email-masked]>
Date: Tue, 29 Mar 2022 11:35:25 +0200

Interesting observations, David. 

This whole thread is probably becoming better for [email-masked]

I think IETF called IETF113 primarily hybrid because it had way too much
remote participation to neglect that they should be given at least more
recognition than in pre-covid remote attendance cases.

Whether IETF113 was indeed hybrid or maybe rather by your scale remote-friendly
is up to the definition of those terms. EXcept for the side-meeting issue,
maybe one could actually call it in between remote-friendly and hybrid. Aka:
there was really very little pain to support remote added to the in-person
experience, but except for maybe the question how to optimize the audio experience
and maybe faster upload of recordings to support other timezones, i am also not
sure how much more could/should have been done for remote attendants.

One thing about the balance is certainly, that an event where a relevant number
of people meeting in-person should have timing set up to best support useful local
day cycles. Which i think we did at IETF113. That to me also says that the
idea of simultaneous in-person set up's across different timezones wouldn't
fly (it was brought up several times in other threads).

Now, the biggest challenge is whether or how one can bring the remote and 
in-person audiences closer together. With all the varying preferences of
participants that certainly the biggest challenge. I am all for hallway table
with gather screen experiments at IETF114, and i am sure we can have many
such experiments over the coming IETF. 

Cheers
    Toerless

-- 
---
[email-masked]

---

From: Pete Resnick <[email-masked]>
Date: Tue, 29 Mar 2022 15:42:03 -0500

> act assertive and respectful (maybe even obnoxious ;-) against them so 

Uh, no. They may have been traditions over time, but (at least for most 
people I know) they are no longer "honored".

If you see this kind of nonsense going on during a WG session, you 
should say so in the chat room, or if you don't feel comfortable doing 
that as a newcomer, you can drop a note to the WG chair(s), and if the 
chair is non-responsive, try the AD. We are really trying to rope in 
this kind of obnoxiousness in WG sessions and elsewhere, and it is the 
chairs' responsibility to do so. If you have particular instances of 
this kind of thing that happened last week, please let chairs or ADs 
know, even now; it's good that they are made aware. If all else fails, 
feel free come to the Ombudsteam. While the Ombudsteam is not supposed 
to be dealing with simple rude behavior in WG sessions, we are always 
willing to communicate things to the right people to make sure that the 
behavior is addressed, and we are required to handle reports completely 
anonymously.

pr
-- 
Pete Resnick https://www.episteme.net/
All connections to the world are tenuous at best