Subject: Re: [vmeet] Remote participation

From: John C Klensin <[email-masked]>
Date: Wed, 28 Mar 2012 04:04:30 -0400

(copying vmeet)

--On Tuesday, March 27, 2012 20:35 -0800 Melinda Shore
<[email-masked]> wrote:

> So far this meeting has been somewhat less remote-friendly
> than usual. Some of it can be attributed to crappy audio and
> it's not clear to me whether and/or to what extent that's
> under the control of the IETF, but while some chairs have done
> an excellent job keeping tabs on remote participants through
> the Jabber rooms (thank you, Paul Hoffman, for doing a great
> job with ipsecme despite funky audio) so far I've found that
> the majority are checking Jabber rooms as an afterthought, if
> at all (granted, probably a sampling problem).  The week isn't
> half over yet, so there's a chance to get things back on track
> and be attentive to your remote participants - make sure that
> audio is working, that they've got an opportunity to provide
> feedback/input, and that someone in the meeting room is
> checking the Jabber room regularly to see if there are issues.

FWIW, I pretty much agree with Melinda.  While I've found the
audio better (when it works at all) than she, SM, and some
others apparently have experienced, input via Jabber seems to
depend on particular individuals (sometimes chairs, sometimes
not) being sensitive to the issues and exerting extra effort.
Webex has been scheduled and not successfully turned on, there
have been long audio outages, Meetecho is very impressive but
seems to have a few nasty bugs (will report separately).  There
are also some other problems that we aren't going to solve at
this meeting but that we'd best get under control sooner rather
than later.

I don't have any theories as to why, but frequency of clearly
audible speaker identification seems to be poorer this time than
in previous years.

Much of this, especially the audio reliability and the Jabber
input situation, seems significantly worse than when I tried to
do Prague remotely a year ago.

I think that part of the problem is the introduction of Etherpad
for notes/online minutes.  That is probably good for the minutes
and record.  However, a year ago (and, I believe, even in Quebec
and Taipei), real-time reporting was mostly on Jabber, which
meant that WGs had active Jabber scribes and a large fraction of
the folks in the room were watching the Jabber room.  This week,
the focus seems to be on Etherpad or other ways of taking notes
and I've been in on sessions in which it has taken effort to get
someone in the room to actually pay attention to Jabber.  If
Chairs and ADs want remote participation, it is _really_
important that you be sure that there is someone in the room who
is monitoring Jabber and sitting close to a microphone.  

   john

---

From: SM <[email-masked]>
Date: Wed, 28 Mar 2012 04:32:20 -0700

Hi John,
>depend on particular individuals (sometimes chairs, sometimes

Audio seems better.  In some sessions, some comments were inaudible.

>audible speaker identification seems to be poorer this time than

Yes.  BTW, that affects minute taking.

>do Prague remotely a year ago.

In some WG sessions, Jabber is barely used.  AD interaction seems to 
affect that.

Given the way the tools are used, remote participants are still the 
Oliver Twists of the IETF.

Regards,
-sm

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 28 Mar 2012 09:28:39 -0400

I think that there should be a detailed checklist available for each
session's chairs. There is
getting to be enough complexity around meetings that I don't think we
should be counting on the
chair's memory to get it all right every time.
(I know that I completely forget about setting up a jabber scribe for
the AntiTrust BOF.) This should IMHO be on line and include things
such as the URLs for webex and other such services.

Regards
Marshall

On Wed, Mar 28, 2012 at 7:32 AM, SM <[email-masked]> wrote:
> In some WG sessions, Jabber is barely used.  AD interaction seems to affect
> that.
>
> Given the way the tools are used, remote participants are still the Oliver
> Twists of the IETF.
>
> Regards,
> -sm
> _______________________________________________
> NOTE WELL: This list operates according to
> http://mipassoc.org/dkim/ietf-list-rules.html.
> https://www.ietf.org/mailman/listinfo/vmeet

---

From: Melinda Shore <[email-masked]>
Date: Wed, 28 Mar 2012 09:07:00 -0800

On 3/28/12 7:59 AM, George, Wes wrote:
> Do we have to write a formal BCP? I'd rather something shorter
 > without all the boilerplate, but I'm not sure if there's a
 > method to do that.

I hope that we haven't reached the point as an organization where we
need BCPs or other formal documents to describe everything we do or
need.  I'll start a page on the chairs wiki.

Melinda

---

From: SM <[email-masked]>
Date: Wed, 28 Mar 2012 11:52:26 -0700

>chair's memory to get it all right every time.

Yes.  Over the last five years, the relevant Wiki page was updated by 
eight persons only.  There is a placeholder for "holding effective WG 
meetings".

At 08:59 28-03-2012, George, Wes wrote:
>[WEG] I've been serving as a Jabber scribe in multiple sessions this 
>week (including in my own WG when we couldn't get a volunteer in a 
>reasonable timeframe). What has occurred to me is that there are 
>really no instructions on *what* one should be doing as a scribe or 
>a note-taker, and so it is intensely variable based on the 
>volunteer. I learned by being in jabber and seeing what others do.

People are expected to volunteer to do X without any instructions of 
what X should be.  It is not surprising that there are very few 
volunteers for note-taking or Jabber scribing.  The same set of 
people end up being volunteered as nobody steps in to help.

>As far as jabber, I've heard more than one chair explain that it 
>means "watch the room for comments, post the link to the slides, 
>notify when they advance to the next slide (especially fun when the 
>slides don't have numbers... chairs??!). It's not clear if jabber 
>scribes are supposed to be identifying who is speaking (for when 
>it's hard to understand the person's name), doing near-realtime 
>transcription/summarization of comments or presenters (when they 
>aren't simply reading the slide bullets to us, that is), or if 
>that's the job of those taking notes.

If the person has participated remotely, he or she might have a sense 
of what information would help.  For example, if the audio quality 
isn't that good, it may help to identify who is at the mic.  Even if 
the audio is good, it's not much work to do that.  It was mentioned 
previously that slides should be numbered.  Sometimes one of two 
sentences about the in-room discussion helps.

There is a Meet and Greet on Sundays.  That's an opportunity to get 
some help as long the volunteer is walked through the experience of a 
session before he or she gets to work.

At 11:01 28-03-2012, Yaron Sheffer wrote:
>Quite subjectively: the audio quality is usually good, and therefore 
>Jabber is mostly useful to "channel in" the remote participants. 
>Asking the Jabber scribe to also summarize what people are saying is 
>putting too high a burden on the scribe, while also hurting both the 
>scribe's and the remote participants' ability to concentrate on 
>what's being said.

It helps for note-taking training.  The burden for a scribe is from 
people who are regular mic attendees.

Instead of projecting slides through the entire session, the Jabber 
room could be projected during discussions.  Currently, Jabber is 
unidirectional with the in-room participants ignoring remote participation.

The log for the TLS session is at 
http://www.ietf.org/jabber/logs/tls/2012-03-28.html  (Jeff is fast at 
the keyboard :-) ).  There seems to have been some problems during 
XMPP ( http://www.ietf.org/jabber/logs/xmpp/2012-03-28.html ).  A 
comparison with HTTPBIS might be interesting ( 
http://www.ietf.org/jabber/logs/httpbis/2012-03-27.html and 
http://tools.ietf.org/wg/httpbis/minutes for etherpad angle).

Regards,
-sm

---

From: Melinda Shore <[email-masked]>
Date: Wed, 28 Mar 2012 11:00:51 -0800

On 3/28/12 10:52 AM, SM wrote:
> Instead of projecting slides through the entire session, the Jabber room
> could be projected during discussions. Currently, Jabber is

I like that a lot.  Pending agreement from my co-chairs I will try that
in Vancouver.

Melinda

---

From: "George, Wes" <[email-masked]>
Date: Wed, 28 Mar 2012 11:59:05 -0400

Melinda] but while some chairs have done
> > the majority are checking Jabber rooms as an afterthought,
[WEG]
<snip>
input via Jabber seems to
> not) being sensitive to the issues and exerting extra effort.

JCK]
> is monitoring Jabber and sitting close to a microphone.

[WEG] I've been serving as a Jabber scribe in multiple sessions this week (including in my own WG when we couldn't get a volunteer in a reasonable timeframe). What has occurred to me is that there are really no instructions on *what* one should be doing as a scribe or a note-taker, and so it is intensely variable based on the volunteer. I learned by being in jabber and seeing what others do.

As far as jabber, I've heard more than one chair explain that it means "watch the room for comments, post the link to the slides, notify when they advance to the next slide (especially fun when the slides don't have numbers... chairs??!). It's not clear if jabber scribes are supposed to be identifying who is speaking (for when it's hard to understand the person's name), doing near-realtime transcription/summarization of comments or presenters (when they aren't simply reading the slide bullets to us, that is), or if that's the job of those taking notes.
We need to decide what the jobs are, and make it clear to volunteers what we expect - something we can point them to when they volunteer that explains it shortly and clearly. There is likely to be some overlap assuming other tools are in place (webex, meetecho, etc), but those probably are the baselines.

Do we have to write a formal BCP? I'd rather something shorter without all the boilerplate, but I'm not sure if there's a method to do that.

Thanks
Wes George

This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.

---

From: Yaron Sheffer <[email-masked]>
Date: Wed, 28 Mar 2012 20:01:51 +0200

Quite subjectively: the audio quality is usually good, and therefore 
Jabber is mostly useful to "channel in" the remote participants. Asking 
the Jabber scribe to also summarize what people are saying is putting 
too high a burden on the scribe, while also hurting both the scribe's 
and the remote participants' ability to concentrate on what's being said.

And to anticipate Melinda's obvious response: when the audio quality is 
very poor I just give up. It's incredibly hard for a remote participant 
to gain anything from such a meeting.

Thanks,
	Yaron

---

From: John C Klensin <[email-masked]>
Date: Wed, 28 Mar 2012 15:11:16 -0400

--On Wednesday, March 28, 2012 11:00 -0800 Melinda Shore
<[email-masked]> wrote:

>> Instead of projecting slides through the entire session, the
>> Jabber room could be projected during discussions. Currently,
>> Jabber is unidirectional with the in-room participants
>> ignoring remote participation.
> 
> I like that a lot.  Pending agreement from my co-chairs I will
> try that in Vancouver.

+1
    john

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 28 Mar 2012 16:22:15 -0400

> unidirectional with the in-room participants ignoring remote participation.

We actually tried doing that with a second projector in several working
groups for a period of a few years.  It worked reasonably well for a
while, but we ultimately gave up, partly because the experiment was
getting too burdensome(*) but also because it wasn't causing people to
pay very much more attention to the jabber room than before.

While I think we (and society in general) do tend to overuse slides in
situations where they don't really add anything, I also think that
dispensing with them entirely is not workable for every session.  So,
short of nearly doubling the number of projectors we own and in some
venues paying expensive fees for additional screens, projecting the
Jabber room would require some way to display both at the same time.

Note that displaying Jabber only "during discussions" may not help,
because the majority of meeting time should be discussions, not
presentations, and I've found that the slides containing the most useful
content are ones used in aiding technical discussions.

-- Jeff

(*) We had to arrange for additional projectors, plus someone had to
bring a machine to run the jabber client for display in front of the
room.  You don't want that to be someone's personal client, because you
don't want personal messages to appear to everyone in the room.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 28 Mar 2012 16:32:49 -0400

I guess it depends upon whether you are holding discussion / question on 
a slide-by-slide basis, or at the end of the presentation.
It strikes me that the best presentations ought to be provoking comments 
on most slides.
And switching back and forth between the Jabber and the slide strikes me 
as awkward and distracting.

Yours,
Joel

PS: In addition, Jeff's later comments about logistics are well-taken.

---

From: John C Klensin <[email-masked]>
Date: Wed, 28 Mar 2012 17:22:14 -0400

--On Wednesday, March 28, 2012 16:32 -0400 "Joel M. Halpern"
<[email-masked]> wrote:

> I guess it depends upon whether you are holding discussion /
> question on a slide-by-slide basis, or at the end of the
> presentation.
> It strikes me that the best presentations ought to be
> provoking comments on most slides.
> And switching back and forth between the Jabber and the slide
> strikes me as awkward and distracting.

Joel,

You are right, of course.  I was thinking about the "wait until
end of session before going to the mic" practice that is common
in plenaries and the "wait until end of presentation before
asking questions" model I've seen in many WGs (probably
appropriate when what is going on is "presentation" rather than
"interactive discussion with slide supplements").

More generally, every time (at least since we gave up on paper
flip charts) I've been involved in a really serious effort to
support either remote participants or local folks with hearing
or language issues, we've ended up with two projectors: e.g.,
slides on one screen, transcript on the other; slides in one
language on one, another on the other; context on one, local
notes on the other.  It isn't hard to think about slides on one
screen, Jabber on the other.   Looking at the way Meetecho uses
screen real estate, I don't imagine this would be terribly hard
(although it might be too much for WebEx... or not).

On the other hand, I'm very sensitive to what Randy said at the
plenary.  There are big advantages of keeping thing simple
rather than getting involved with more complex approaches that
take longer to implement, create more opportunities for things
to go wrong, and possibly require more support.   Unless I
misunderstand it, the suggestion to project the Jabber session
was, in part, a way of getting around a problem we shouldn't
have in the first place.  To state this far more extremely than
I think would actually be necessary, suppose that the IESG took
the position that any WG Chair who couldn't manage discussion
flows sufficiently well to be sure that all speakers clearly
announced their names* and questions and comments asked via
Jabber were picked up and integrated reasonably quickly was not
up to the job and, after a warning or two, would be replaced.  I
think a lot of the problem would go away quickly.

That doesn't mean that projecting the Jabber room for better
interaction during a long discussion wouldn't be a good idea --
I think it would be.  But my guess is that the "everyone talks,
a few people type" model wouldn't scale very well.  Remember
that even a good conference-grade speaker phone in the middle of
a table works quite well at sufficiently small scale even though
it doesn't work well at all in a large room with lots of people.

      best,
        john

p.s. Those who are enthused about use of barcodes or RFID to
identify speakers should note that, in addition to the issues
mentioned at the plenary, there is still a requirement for
real-time written or spoken identification of speakers for
remote users and those who have trouble hearing.  That means
that, unless we invent yet more technology to feed the names to
remote users (more complexity with almost no payoff) or want to
amuse ourselves by finding out how much of a fool a speech
synthesizer can make of its virtual self on "foreign" names (to
pick on the address list for this message, contemplate
"Subramanian", "Klensin", or even figure out how whether "Joel"
is pronounced as one syllable or two)... well, maybe a real-time
screen in front of the Jabber scribe who then gets to watch the
big screen, the mike line, that screen, and sometimes the
keyboard and his or her own screen at the same time.     Nah.

---

From: SM <[email-masked]>
Date: Wed, 28 Mar 2012 15:11:45 -0700

>pay very much more attention to the jabber room than before.

I left out the burdensome points you mentioned.  I don't think there 
is a solution to all of them.  I avoided mentioning a second 
projector for cost and also as it's one more thing to focus 
on.  There is also the question of the jabber room conversation not 
being a side meeting within the session.

>content are ones used in aiding technical discussions.

To add to what you said, the Jabber display also depends on the size 
of the session.  Someone remote asks whether he counts.  As the 
answer can end up being problematic, I won't comment on that.

As a strawman's guideline, I'll suggest 13 minutes for Jabber 
projection in a one hour session.  If a slide is useful to aiding the 
technical discussion, it takes precedence.  It's possible to get a 
sense of slide timing from http://www.ietf.org/jabber/logs/soc/2012-03-28.html

At 13:32 28-03-2012, Joel M. Halpern wrote:
>question on a slide-by-slide basis, or at the end of the presentation.
>It strikes me that the best presentations ought to be provoking 
>comments on most slides.

Yes.  I also left out that the jabber projection may not be visible 
from the back of a room.

Melinda Shore outdid herself with 
http://wiki.tools.ietf.org/group/wgchairs/wiki/RemoteParticipation

Regards,
-sm

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 28 Mar 2012 19:24:25 -0400

On Wed, 2012-03-28 at 15:11 -0700, SM wrote:

> on.

It is, however, something that's been tried.  I do hope that some of the
current and former chairs of KITTEN, KRB-WG, and SASL who are in Paris
(which, unfortunately, I am not) will attend the rpsreqs BOF session on
Friday afternoon and/or otherwise get in touch with Paul to share our
experiences with that.

> of the session.  Someone remote asks whether he counts.

I'm not sure what that means.  Certainly remote participants should have
just as much right to be heard as those who are present, and should be
given just as much weight in determining whether there is a consensus.
I for one consider it to be a key principle of the way the IETF carries
out its work that participating in the decision process does not require
physical attendance at meetings.

> http://wiki.tools.ietf.org/group/wgchairs/wiki/RemoteParticipation

Yes, that's a very nice job.

FWIW, when I am physically present to chair a session, I normally try to
verify in advance that all of the mics are present and the mix isn't too
terrible.

-- Jeff

---

From: SM <[email-masked]>
Date: Thu, 29 Mar 2012 00:17:57 -0700

Hi Jeff,
>I'm not sure what that means.  Certainly remote participants should have

My guess is whether the opinion of the remote person will be given 
any consideration.

>physical attendance at meetings.

This comment is not directed at you.

The above is well-stated.  There have been questions, sometimes from 
people attending in person, about whether what is mentioned about is 
just for show only.  The questions go unheard.  I take it that the 
key principle can be explained by:

   (a) There is an audio stream.

   (b) There is a Jabber room.

   (c) There is a Jabber scribe.

If any of these above is not available, would the session chair 
cancel the meeting?

There have been some comments about scheduling yesterday.  Would WG 
chairs object if the Area Director cancels a session if the chair has 
not publicly announced two weeks before the session the names of the 
persons who will act as Jabber scribe and minute-taker?  That might 
free up a few slots.

>verify in advance that all of the mics are present and the mix isn't too

+1.

Regards,
-sm

---

From: Tony Hansen <[email-masked]>
Date: Thu, 29 Mar 2012 06:50:59 -0400

There are groups that regularly project the jabber conversation. The 
secretariat is actually willing to provide a 2nd projector if you ask 
nicely.

There are several separable things that are currently commonly conflated 
via the back end channel: taking notes/minutes, keeping a running  
indication of what slide is currently being presented, having real-time 
transcription of what was said, a forum for side-comments, and as a 
channel for remote participants. Having *all* of them on a single jabber 
channel is messy and makes it much harder for remote participants to 
actually participate -- even someone watching the jabber log diligently 
for comments to be channeled can get confused as to which things need 
channeling and which things are the many other things going on that channel.

One WG I was involved with this week effectively used the etherpad 
minutes taker for doing the notes/minutes and indicating who was 
talking. This removed all of that stuff from the jabber stream and 
allowed the jabber to be a more effective channel mechanism.

     Tony

---

From: Dave Crocker <[email-masked]>
Date: Thu, 29 Mar 2012 13:16:12 +0200

On 3/29/2012 12:50 PM, Tony Hansen wrote:
> There are several separable things that are currently commonly conflated via the
> back end channel: taking notes/minutes, keeping a running indication of what
> slide is currently being presented, having real-time transcription of what was
> said, a forum for side-comments, and as a channel for remote participants.
> Having *all* of them on a single jabber channel is messy and makes it much
> harder for remote participants to actually participate -- even someone watching
> the jabber log diligently for comments to be channeled can get confused as to
> which things need channeling and which things are the many other things going on
> that channel.
>
> One WG I was involved with this week effectively used the etherpad minutes taker
> for doing the notes/minutes and indicating who was talking. This removed all of
> that stuff from the jabber stream and allowed the jabber to be a more effective
> channel mechanism.

Meetecho combines jabber with presentation slides.  jabber can be selectively 
displayed.  (Etherpad remains separate.)

This conveniently permits a single display to contain the information streams of 
slides and remote postings.  (however it does effectively require all slides be 
uploaded to a single, controlling machine.)

For my small working group (repute) with an agenda that was not tightly packed 
and with a small number of remote participants, it was convenient to keep jabber 
showing and to have an open room 'chat' among /all/ participants.  That is, we 
did not need the time or organization discipline that comes with tight, 
slide-based presentations and/or many active participants at the microphone.

For the more typical case that does need more session management, we are used to 
having someone speak for a jabber participant.  What the Meetecho arrangement 
allows is, instead, keeping jabber off the screen, except when it's a jabber 
poster's turn, and then display their text rather than having someone running up 
and down at the mic channeling them.

There are more conveniences to the jabber/slide integration, since the next 
slide displayed has its number and title injected into jabber, thereby 
synchronizing the jabber record nicely.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Al Morton <[email-masked]>
Date: Thu, 29 Mar 2012 08:58:56 -0400

At 06:50 AM 3/29/2012, Tony Hansen wrote:
>...One WG I was involved with this week effectively used the 
>etherpad minutes taker for doing the notes/minutes and indicating 
>who was talking. This removed all of that stuff from the jabber 
>stream and allowed the jabber to be a more effective channel mechanism.
+1, did this twice in Paris and once in Vancouver, with great success.
Al

---

From: Steven Bellovin <[email-masked]>
Date: Fri, 30 Mar 2012 09:20:37 +0200

On Mar 28, 2012, at 10:22 PM, Jeffrey Hutzelman wrote:
> situations where they don't really add anything,

I'll note that folks who are non-native speakers of English are often
helped significantly by having slides projected.  I've certainly
seen many people taking pictures of the screen, back before early
upload was the policy.

		--Steve Bellovin, https://www.cs.columbia.edu/~smb

---

From: John C Klensin <[email-masked]>
Date: Fri, 30 Mar 2012 11:48:28 -0400

--On Friday, March 30, 2012 09:20 +0200 Steven Bellovin
<[email-masked]> wrote:

>...
> I'll note that folks who are non-native speakers of English
> are often helped significantly by having slides projected.
> I've certainly seen many people taking pictures of the screen,
> back before early upload was the policy.

Agreed, but I'd like to expand on this a bit.

(1) Almost any observation that is based on the advantages of
slides by non-native speakers also applies to real-time minutes,
transcripts, Jabber sessions, etc.  Done well, any or all of
them may be easier to understand than a real-time speaker,
especially one who is speaking too quickly.

(2) The symmetric situation applies as well.  We have presenters
who speak too rapidly, have accents or styles of speaking
English that are hard to understand, and so on.  Having good
slides, etc., available may considerably facilitate
understanding, even (or especially) by native speakers of
English and, in some cases, by native speakers of some other
language than that of the presenter.

In both cases, a comment I made in the Jabber room may be
relevant: seeing the slides in real time may be great (and may
be very important for knowing when and how to ask questions),
but, if one is trying to use the slides to help in understanding
rather than merely to help with or keep context, the ability to
scroll backwards (and even forwards) may be a critical feature.
Of course doing that interacts with comments about how much is
happening on the screens of such participants, how many windows
one needs to watch, etc.   As a trivial example, an application
that thinks it is so important that it has to seize focus and
maybe take the screen over every time something happens (new
slide, new message, etc.) can rather quickly turn into a
cognition-impairing nightmare.

    john

---

From: SM <[email-masked]>
Date: Fri, 30 Mar 2012 11:34:22 -0700

>especially one who is speaking too quickly.

If I recall correctly, a very rough estimate of the cost for 
providing real-time transcription of one week of sessions came to 
under $10,000.

Advantages are:

   (a) The discussions are more accessible for non-English (local and
       remote) participants

   (b) Local participants in another session can follow what is going on

   (c) The transcript can be used to produce minutes for the session

   (d) The transcript can be read by people who have not attended the session

   (e) The transcript can be (easily) translated by the reader

   (f) It does not require an additional screen

   (g) A minute-taker is no longer required

   (h) Less pressure to upload slides before the session

   (i) The content of a session is searchable

Disadvantages are:

   (a) It is not always possible for the person transcribing to capture
       the name of the speaker or some technical words

   (b) It's an additional cost

   (c) There can be network connectivity issues

   (d) There may be an indirect negative impact on attendance revenue

   (e) It may be workable for regular meetings only

   (f) The content of a session is searchable

 From the IETF 82 proceedings:

   Technical plenary http://www.ietf.org/proceedings/82/minutes/plenaryt.txt
   AppArea           http://www.ietf.org/proceedings/82/minutes/appsawg.txt
   IntArea           http://www.ietf.org/proceedings/82/minutes/intarea.txt
   OpsArea           http://www.ietf.org/proceedings/82/minutes/opsawg.txt
   RtgArea           http://www.ietf.org/proceedings/82/minutes/rtgwg.txt
   SecArea           http://www.ietf.org/proceedings/82/minutes/saag.txt
   TsvArea           http://www.ietf.org/proceedings/82/minutes/tsvwg.txt

The quality of reporting varies widely.  I suggest reading the 
proceedings for an area you are involved in to get an idea of the 
remote participant experience.  For English as a foreign language, 
pick an area you are not involved in.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Fri, 30 Mar 2012 18:14:56 -0400

--On Friday, March 30, 2012 11:34 -0700 SM <[email-masked]>
wrote:

>> slides by non-native speakers also applies to real-time
>> minutes, transcripts, Jabber sessions, etc.  Done well, any
>> or all of them may be easier to understand than a real-time
>> speaker, especially one who is speaking too quickly.
> 
> providing real-time transcription of one week of sessions came
> to under $10,000.

Just so we understand what we are talking about, that means all
eight tracks, plus the plenaries?  My recollection of how much
skilled transcribers can do at a time combined with the length
of our day, is that would mean at least 10 and probably 12-15,
transcription people.  Is that roughly what the figure you give
is based on?  And does it include airfare and hotels for those
people (noting that they would, e.g., outnumber Secretariat
staff for the duration of a meeting.

> Advantages are:
> 
>    (a) The discussions are more accessible for non-English
> (local and remote) participants
> 

It does if people who are in the room want to see the
transcript.  Experience at multiple IETF plenaries and several
other organizations (including ICANN) indicates that they
usually do.  The other advantage of transcripts is that they can
be used by folks who don't hear well.  If they are in the room,
that certainly requires an extra screen, at least unless you
expect that population to all bring laptops and spend their time
hunched over them.

>    (h) Less pressure to upload slides before the session

I don't think so.  For non-native speakers of whatever language
the speakers are speaking, ability to review slides in advance
is a big help.

>        the name of the speaker or some technical words

If the speaker doesn't speak English really clearly, or speaks
it very quickly, or both, the quality of transcription can
easily fall below the level needed to make the above advantages
real.  In that case, the problem is not just speaker names and
technical words

best,
   john

---

From: SM <[email-masked]>
Date: Fri, 30 Mar 2012 16:18:57 -0700

Hi John,
>eight tracks, plus the plenaries?  My recollection of how much

Yes.

>staff for the duration of a meeting.

No, I meant doing the transcription remotely to avoid the expenses of 
airfare and hotels.  Skilled transcribers are more expensive.  BTW, I 
looked up the figures over a year ago.

>expect that population to all bring laptops and spend their time

The screen may not be as visible from the back of the room.  If 
people in the room want to see the transcript, they bring their 
laptop.  If you can follow the conversation, you don't need the 
transcript.  The service could be optional; it's up to the session 
chair to make the call.

>the speakers are speaking, ability to review slides in advance

Ok.

>real.  In that case, the problem is not just speaker names and

Yes.  If the quality of transcription falls below an acceptable 
level, the question that arise is whether the people in the room have 
understood what they heard.  But let's not go there as we'll end up 
trying to solve a lot of problems.

After reading 
http://www.ietf.org/mail-archive/web/oauth/current/msg08692.html my 
opinion is that having a transcription might help in such cases.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Fri, 30 Mar 2012 22:55:59 -0400

--On Friday, March 30, 2012 16:18 -0700 SM <[email-masked]>
wrote:

>> Just so we understand what we are talking about, that means
>> all eight tracks, plus the plenaries?  My recollection of how
>> much
> 
> Yes.
> 
>> transcription people.  Is that roughly what the figure you
>> give is based on?  And does it include airfare and hotels for
>> those people (noting that they would, e.g., outnumber
>> Secretariat staff for the duration of a meeting.
> 
> No, I meant doing the transcription remotely to avoid the
> expenses of airfare and hotels.  Skilled transcribers are more
> expensive.  BTW, I looked up the figures over a year ago.

Interesting idea.  Of course, if the audio fails, we would
essentially have what used to be called a domino effect: if
minutes and everything else depended on the transcribers, the
transcribers were remote, and the audio went out, the WG might
reasonably adjourn to the nearest location dispensing coffee or
stronger beverages.

>> usually do.  The other advantage of transcripts is that they
>> can be used by folks who don't hear well.  If they are in the
>> room, that certainly requires an extra screen, at least
>> unless you expect that population to all bring laptops and
>> spend their time hunched over them.
> 
> The screen may not be as visible from the back of the room.
> If people in the room want to see the transcript, they bring
> their laptop.  If you can follow the conversation, you don't
> need the transcript.  The service could be optional; it's up
> to the session chair to make the call.

I worry a little about expecting people to read transcripts on
their laptops for three reasons:

	(i) Trying to keep track of the laptop screen largely
	prevents looking at the slides which, we would like to
	hope, actually contain information.  It might interfere
	with Jabber tracking even more.
	
	(ii) It might discourage getting up and standing in
	microphone lines, etc., to comment and participate.
	
	(iii) For a WG Chair (or equivalent) looking into the
	audience and seeing who is huddled over their laptops,
	presumably reading mail and who is paying attention.
	Having a significant number of people huddled over their
	laptops reading transcripts fouls up that particular bit
	of inference because that group is mentally present in
	the WG, while most of the mail-readers are not.

>...
> After reading
> http://www.ietf.org/mail-archive/web/oauth/current/msg08692.ht
> ml my opinion is that having a transcription might help in
> such cases.

I can't dispute your opinion.

regards,
   john

---

From: Marshall Eubanks <[email-masked]>
Date: Sat, 31 Mar 2012 04:19:27 -0400

Dear John;

> reasonably adjourn to the nearest location dispensing coffee or

I think that we may be moving towards a regime where that would be true
in any case. At some point, as the number of remote participants grows
(including participants who rely on virtual attendance), the ability to have
an open standards organization meeting will depend on the technology working.

Suppose, for example, you had a WG meeting where most of the
proponents of technology X were local, and physically present, and the
proponents of technology Y were foreign, and virtually present. If the
remote participation technology fails, can the WG Chair legitimately
claim "in-room" consensus to select technology X? (Of course, such
in-room consensus would have to be confirmed on the WG mailing list,
but  meeting consensus can have a big influence on mailing list
consensus.)

The result of what we are doing here will I think inevitably at some
point change the answer to no. And, after that, you are not having a
WG meeting, just some sort of seminar.

Regards
Marshall

---

From: SM <[email-masked]>
Date: Sat, 31 Mar 2012 03:49:07 -0700

Hi John,
>reasonably adjourn to the nearest location dispensing coffee or

If the audio fails, you have audio recordings to work from.  It's up 
to the session chair to assess the situation and figure out what to 
do.  It is reasonable for the WG to adjourn.  It is also reasonable 
for the AD to shut down the WG if it's not delivering.

Yes.

>         microphone lines, etc., to comment and participate.

The transcription might provide some statistics about mic attendees, 
how much time they spend at the mic, whether the comment actually 
helps to resolve the issue being discussed.

In a session on Friday, overall, the people who stepped to the mic 
line are from North America.  There was only one non-English 
speaker.  Around 83% of the IETF output is from English 
speakers.  That may fall down to around 60% in the near future.  If 
we ignore the presentations, the numbers for standing up in 
microphone lines do not tally with the trend.

Nothing in what I suggest is going to fix that.

>         the WG, while most of the mail-readers are not.

There is a significant number of tourists in a WG session.  If a 
person is not paying attention, he/she shouldn't be surprised when 
the minutes shows that he/she contributed to some decision.

BTW, I forgot to mention that some of the tourists are a bit more 
than that.  They take notes of what goes on in a session for internal 
reporting.

Yes.

>WG meeting, just some sort of seminar.

If we exclude WG adoption of drafts, how many decisions are taken (to 
be confirmed on the list) in a meeting?

  OAUTH     5
  MANET     4
  TLS       3
  KITTEN    2
  DHC       2
  P2PSIP    2
  SIPCORE   2
  XRBLOCK   2
  WEBSEC    1
  ANTITRUST 1
  INSIPID   1
  KERBEROS  1
  ISIS      1
  JOSE      0
  IPSECME   0
  PKIX      0
  MILE      0
  NEA       0
  ABFAB     0
  DANE      0

The metrics used do not paint a true picture of the sessions.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Sat, 31 Mar 2012 22:46:30 -0400

--On Saturday, March 31, 2012 04:19 -0400 Marshall Eubanks
<[email-masked]> wrote:

>>> expenses of airfare and hotels.  Skilled transcribers are
>>> more expensive.  BTW, I looked up the figures over a year
>>> ago.
>> 
>> reasonably adjourn to the nearest location dispensing coffee
>> or stronger beverages.

> I think that we may be moving towards a regime where that
> would be true in any case. At some point, as the number of
> remote participants grows (including participants who rely on
> virtual attendance), the ability to have an open standards
> organization meeting will depend on the technology working.
> 
> proponents of technology X were local, and physically present,
> and the proponents of technology Y were foreign, and virtually
> present. If the remote participation technology fails, can the
> WG Chair legitimately claim "in-room" consensus to select
> technology X? (Of course, such in-room consensus would have to
> be confirmed on the WG mailing list, but  meeting consensus
> can have a big influence on mailing list consensus.)
> 
> The result of what we are doing here will I think inevitably
> at some point change the answer to no. And, after that, you
> are not having a WG meeting, just some sort of seminar.

Marshall,

I think we are on the same page.

    john