Subject: WebEX in WG Sessions

From: Russ Housley <[email-masked]>
Date: Tue, 13 Apr 2010 16:09:09 -0400

Some WGs have been making use of WebEX to allow remote participants to
make presentations.  It also allows remote participants to listen to the
audio stream and see the slides that are being shown in the room.  The
popularity of the WebEX service has grown from IETF76 to IETF77.  If
this growth continues, then we will be placing too great a burden on the
volunteers that have been making this service available.  I am
considering a new requirement for a WG to use WebEX, and I want to hear
your thoughts before I take it to the IESG.  Of course, IESG members are
on this mail list, so they will see this discussion, so they know it is
coming.

To reduce the burden on the WebEX volunteers, I am considering asking
each WG that wants to use WebEX during their meeting session to have two
people other than the WG Chairs trained on the WebEX setup on Sunday.
We have found that running the meeting and running WebEX is too great a
burden for the WG Chair; we need the WG Chair to focus on the meeting,
not the WebEX tools.   The trained people will be responsible for
starting the WebEX session, monitoring it during the WG meeting, and
then shutting it down at the end of the WG meeting.  If problems are
encountered, these trained people will be able to call on the WebEX
volunteers for help.  In this way, the WebEX volunteers can focus on
trouble spots.

Thoughts?

Russ

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 13 Apr 2010 16:14:41 -0400

Sounds like a good idea.
Joel

---

From: Joe Touch <[email-masked]>
Date: Tue, 13 Apr 2010 13:17:27 -0700

Hi, Russ,

AFAICT, it's challenging enough to get scribes and jabber scribes.
Asking for WebEx coordinators who arrive on Sunday for training seems
infeasible, IMO.

Joe

---

From: Brian Rosen <[email-masked]>
Date: Tue, 13 Apr 2010 16:19:46 -0400

I agree that WG chairs can't run the Webex session.  We need volunteers.
OTOH, getting two volunteers recruited before the session, and getting them
there for training on Sunday is probably not going to work.

Do we really need two?  Seems like one is enough.

On the other, other hand, finding people who already know how to run WebEx
is getting easier all the time.

So, I don't think requiring local volunteers to run it (as we do for note
taker and jabber scribe) is any burden, but actually requiring training is.

I am worried about how many volunteers we are getting into.  Seems like we
could combine roles.

Brian

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 13 Apr 2010 13:21:23 -0700

During this training, it would be great if there were hints on how to make the folks speaking over WebEx more audible. In the IPsecME session in Anaheim, the echoes really bit us until I unplugged the fixed mic for the chairs *and* turned off the roving mic. I suspect it would have been better if we turned off the audience mic as well, but I didn't want to hassle with that.

This kind of training / experimentation could make the WebEx experience much better.

--Paul Hoffman, Director
--VPN Consortium

---

From: Sean Turner <[email-masked]>
Date: Tue, 13 Apr 2010 16:24:14 -0400

This sounds like a good job for a WG secretary.

spt

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 13 Apr 2010 16:28:18 -0400

If you can't get the volunteers, then don't use WebEX for your session.

Yes, we all have trouble getting volunteers.  For everything.
It is not like the overall organization has access to volunteers we 
don't have.  Asking the working group to help support the labor 
necessary to use a tool we want to use seems fair and appropriate.

Do we as chairs expect Russ to provide us with jabber scribes?  While it 
would be nice, it just isn't going to happen.

Yours,
Joel

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 13 Apr 2010 22:31:18 +0200

I'd feel much more comfortable to read this message after an s/WebEX/$PRODUCT/g.
Are we really stuck with a mechanism that is so labor-intensive that it *requires* training (and consuming the attention of!) two facilitators per meeting?  Who is supposed to do this, if it detracts them from the meeting so much?  Clearly, facilitators are a good idea, but I have this nagging feeling that we are making a technological choice here that is a significant regression after all the wonderfulness we got out of vat/rat, wb, and nte in 1990s.

Just a thought.

But then, independent of remote participation, I'd love to have an audio watchdog at every meeting that physically forces people to properly attach/use their microphones, not pop into them, speak at an intelligible level and speed.  The chairs could use a slide advancer that knows how to advance both PPT and PDF (and a slide watchdog that makes sure these are readable and available in the MMM in the first place).  And a discussion-summarizer that makes sure important points are visible on the projectors.  And so on.  I'm not against facilitators, just mildly opposed to overloading their attention with another difficult-to-use tool.

Gruesse, Carsten

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 13 Apr 2010 16:31:42 -0400

> people other than the WG Chairs trained on the WebEX setup on Sunday.

Despite the difficulty of recruiting volunteers for sessions, I think
this is a good idea, as long as it's a policy that is subject to
review all the time.  (It might be that the technology will improve
sufficiently that things will become easy enough that people don't
need training, or that remote software of this sort in our line of
work becomes so ubiquitous that basically everyone will be able to do
it.)  But for the time being, if we're going to support this kind of
remote participation, we need really to support it.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Joe Touch <[email-masked]>
Date: Tue, 13 Apr 2010 13:35:19 -0700

> would be nice, it just isn't going to happen.

I do understand this viewpoint, however this isn't just "yet another
volunteer"; we're being asked to provide someone who will be there
Sunday afternoon for training.

That's two orders of magnitude harder to find (IMO) than just another
volunteer at a meeting.

Joe

---

From: David Sinicrope <[email-masked]>
Date: Tue, 13 Apr 2010 16:47:16 -0400

WG secretaries are normally taking minutes and can't share their  
screens for the meetings.

A couple other points from when we used it in PWE3:
- for co chairs it seems to help when one chair shares the screen and  
the other keeps the agenda up
- the co chair or secretary can join the webex to get the participant  
list so we can see whose using it

------------------
David Sinicrope

---

From: Marc Blanchet <[email-masked]>
Date: Tue, 13 Apr 2010 16:48:37 -0400

0) I think the requirement is remote participation, not necessarily 
webex. webex is one tool to acheive this.
a) webex seems complicated to setup.
b) from what I've heard before and agree, audio feed (maybe even one 
way), jabber and slides sent in advance does a pretty good job. jabber 
relays for questions.
c) another idea: back to a more simple setup as described in b), voip 
based, that would require almost no training, much easier setup, ...

Marc.

-- 
=========
IPv6 book: Migrating to IPv6, Wiley. http://www.ipv6book.ca
Stun/Turn server for VoIP NAT-FW traversal: http://numb.viagenie.ca
DTN news service: http://reeves.viagenie.ca
NAT64-DNS64 Opensource: http://ecdysis.viagenie.ca

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Tue, 13 Apr 2010 22:49:12 +0200

I can support that approach

Bert
  ----- Original Message ----- 
  From: Russ Housley 
  To: IETF WG Chairs 
  Sent: Tuesday, April 13, 2010 10:09 PM
  Subject: WebEX in WG Sessions

  Thoughts?

  Russ

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 13 Apr 2010 15:11:38 -0600

There are a few functions that webex could provide for IETF meetings. I see the major ones as 

1) sharing the slides to remote participants 

2) allowing remote folks to hear what is happening in room 

3) allowing remote folks to "speak" to the room 

4) text chat 

Consider each of these in turn....

1) Obviously download the slides also works - but presenters forget to say "on slide 7", so it is hard to know which slides is currently displayed. I see some value in webex for this this but having the jabber chat room indicate the slide works about as well and removes some of the snafus of which machine is being presented off of and webex "reformatting" slides

2) The streaming media is sometimes easier to understand but the latency is higher too. It might be nice to adjust the coding to have less latency and very high audio quality.  

3) This was the big value prop of using webex. In my mind it has not worked and we should be careful about expanding webex use until a solution is found that works. The issues of room echo and echo cancelation have not yet been solved in the way we are using webex. How we do floor control is not really been figured out yet. Currently remote people that want to participate have better luck just using the jabber channel and having someone read their question at the mic. When that beats your audio solution, you know the audio solution needs some work. 

4) The WG jabber chat rooms work better for this.

Overall if people have new ideas to make the remote audio actually work, I'd like to see some experiments with theses ideas but the current situation is not really solving 3. 

The remote presentation over webex have been a waste of WG time and I would recommend people don't do that. In every remote presentation I have seen, the presenting person could not actually answer questions from the room due to the audio being so messed up.

more inline ...

On Apr 13, 2010, at 2:09 PM, Russ Housley wrote:

> then shutting it down at the end of the WG meeting.

So who's computer are you going to present off of? 

>  If problems are

Cullen Jennings
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Yaron Sheffer <[email-masked]>
Date: Wed, 14 Apr 2010 00:16:05 +0300

During IETF-77 I had the experience of being on the "wrong" side of
WebEX - trying to give a presentation using WebEx (through a PSTN
connection) for bidirectional audio. We did not use it for slides or for
chat. According to the feedback I received, audio quality in the room
was miserable, which rendered the presentation less than effective.

Earlier I had a bad experience trying, and failing, to install WebEx on
a Windows laptop. Ironically, that was during the WebEx training
session.

Putting aside WebEx-specific application issues, bidirectional audio
still seems to require heavy duty equipment and/or audio experts. So my
conclusions are in support of Marc's simplified setup, with only the
existing one-way audio streaming.

If better technology is available today (in particular echo
suppression), I would love to be proven wrong.

Thanks,
	Yaron

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 13 Apr 2010 16:38:55 -0500

OK, at the risk of having this discussion on a strange mailing list (wasn't 
this what https://www.ietf.org/mailman/listinfo/vmeet was intended for?)...

Re: 1 - yeah, it would be AWESOME if people announced slide changes 
audibly...

Re: 2 and 3 - In Hiroshima, we had remote participants in Europe who were 
experiencing excellent streaming audio quality with 10-second delays - it 
was great for lurkers, but we couldn't come close to using it for an 
interactive discussion with a remote participant (using jabber the other 
direction). I think interactive discussion with remote participants is the 
problem WebEx is really supposed to help with, compared to our audio 
streaming.

I'm assuming that it's better if you're in Ansheim and your remote 
participant is in San Francisco, but I don't know that.

Re: 4 - Cullen, I know that the IAB still uses a jabber room for WebEx 
telechats - does the IESG do the same thing? So I don't think people are 
that impressed with text chat on WebEx.

Thanks,

Spencer

----- Original Message ----- 
From: "Cullen Jennings" <[email-masked]>
To: "Russ Housley" <[email-masked]>
Cc: "IETF WG Chairs" <[email-masked]>
Sent: Tuesday, April 13, 2010 4:11 PM
Subject: Re: WebEX in WG Sessions

There are a few functions that webex could provide for IETF meetings. I see 
the major ones as

4) text chat

1) Obviously download the slides also works - but presenters forget to say 
"on slide 7", so it is hard to know which slides is currently displayed. I 
see some value in webex for this this but having the jabber chat room 
indicate the slide works about as well and removes some of the snafus of 
which machine is being presented off of and webex "reformatting" slides

2) The streaming media is sometimes easier to understand but the latency is 
higher too. It might be nice to adjust the coding to have less latency and 
very high audio quality.

3) This was the big value prop of using webex. In my mind it has not worked 
and we should be careful about expanding webex use until a solution is found 
that works. The issues of room echo and echo cancelation have not yet been 
solved in the way we are using webex. How we do floor control is not really 
been figured out yet. Currently remote people that want to participate have 
better luck just using the jabber channel and having someone read their 
question at the mic. When that beats your audio solution, you know the audio 
solution needs some work.

Overall if people have new ideas to make the remote audio actually work, I'd 
like to see some experiments with theses ideas but the current situation is 
not really solving 3.

The remote presentation over webex have been a waste of WG time and I would 
recommend people don't do that. In every remote presentation I have seen, 
the presenting person could not actually answer questions from the room due 
to the audio being so messed up.

more inline ...

Cullen Jennings

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 13 Apr 2010 22:53:30 +0100

Without having tried it for the WG I chair, I have
the impression from other sessions that I've sat
in on that this is more disruptive than useful, esp
the remote audio into the room (as Cullen says).

That has also been my experience of remote input
in various other situations. Lurking and jabber do
seem to work fine, but active better-than-text
remote interaction with a large space/audience still
seems to me like an unsolved problem.

So, I'd be for trying other things or playing more
with this set of tools before investing much in
them. In particular I think it'd be premature to
include this in Sunday training, or as a requirement
for any WG or role.

OTOH, I've no objection if others want to do (more
of) this so long as we're not creating a situation
where its (one way or another) forced on a WG that
don't want or need it.

Stephen.

PS: Maybe force bar-BoFs with no beer to use this:-)

---

From: Tony Li <[email-masked]>
Date: Tue, 13 Apr 2010 14:58:51 -0700

> participant is in San Francisco, but I don't know that.

Data point: I regularly use WebEx for multi-party phone-to-phone chats,
including half-way around the world.  It works quite well and quite
consistently.  There are always issues with background noise (kids
screaming, vacuum cleaners, etc.) but this will happen with any
bi-directional solution.  It works with conference room systems (i.e.,
Polycom).

If other folks corroborate this, then I put forth the hypothesis that the
real issues are all about audio _in the room_.   I say this not to point
blame, but to help with possible troubleshooting.

Regards,
Tony

P.s.  Disclaimer: Yes, WebEx is Yet Another Subsidiary of my current
employer.  This comment is made purely as a personal contribution.  I have
no relationship with the WebEx side of the house, other than as a (mostly)
satisfied customer.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 13 Apr 2010 18:01:32 -0400

There really are two separate issues.
Issue 1) Is WebEX the best tool we have at the moment for remote 
participation, and do we want to use.  WG meetings are not required to 
use WebEX.  They are expected, but not required, to use jabber.
Issue 2) If we are going to use WebEX, where are the necessary resources 
going to come from.

Please do not cross the debates.
I have never used WebEX for IETF work.  I hav eno idea how well it works 
for that.  I will defer to others on issue 1.

My comment was about, and I believe Russ question was about, issue 2.

(Yes, there is a weak linkage in that we are finding that WebEX takes 
more resources than we though.  That may be a reason to just give up on it.)

Yours,
Joel

---

From: Russ Housley <[email-masked]>
Date: Tue, 13 Apr 2010 18:02:56 -0400

> screens for the meetings.

The Secretariat provides a laptop that runs the WebEX application.

I suspect the WG Secretary will be focused on notes for the minutes.

Russ

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 13 Apr 2010 17:04:59 -0500

Hi, Tony,

> bi-directional solution.  It works with conference room systems (i.e.,

Heh, heh... yeah. One of the working groups I co-chair (with Eric Berger) is 
MEDIACTRL. In Hiroshima, after we figured out what our streaming audio 
latency really looked like, some of our participants brought up their own 
conference bridge and we used that for two-way audio, with no pain at all.

So your recent WebEx experience on delay matches my own. The long delays we 
were experiencing were specifically with the IETF audio streams.

Thanks,

Spencer

---

From: Russ Housley <[email-masked]>
Date: Tue, 13 Apr 2010 18:09:02 -0400

> webex. webex is one tool to acheive this.

Right.  But, it is the one that we are experimenting with right now
because of the generous donation from Cisco.

> a) webex seems complicated to setup.

It is complex because the sound system in the meeting room needs to be
connected to the WebEX session.  This allows the WebEX participants to
hear what is said in the room and allows the people in the room to hear
the WebEX participants.

> way), jabber and slides sent in advance does a pretty good job. jabber

This works for remote participates to follow what is happening in the
meeting room.  WebEX also allows remote presentations.  The other does not.

> based, that would require almost no training, much easier setup, ...

No.  The solution being used is VoIP based.

Russ

---

From: <[email-masked]>
Date: Wed, 14 Apr 2010 02:04:02 +0200

Internally, we use to rely on WebEX or similar solutions for remote participation to a meeting.
If all the participants are remote, no problem. Problems appear when remote participants are in the minority compared to the interested people in the meeting room. Floor control is especially quite challenging, even if the audio channel works well.

I share the idea that we could first ensure that slides are provided right in time before the meeting, ensure a good quality of the audio stream and encourage the use of the jabber room for exchange. About remote presentation, I'm afraid that the interactive part following the presentation will be too often time-consuming to be really efficient.

Regards,

Lionel

> -----Message d'origine-----
> De : [email-masked] 
> [mailto:[email-masked]] De la part de Russ Housley
> Envoyé : mercredi 14 avril 2010 00:09
> À : IETF WG Chairs
> Objet : Re: WebEX in WG Sessions
> 
> 
> Right.  But, it is the one that we are experimenting with 
> right now because of the generous donation from Cisco.
> 
> It is complex because the sound system in the meeting room 
> needs to be connected to the WebEX session.  This allows the 
> WebEX participants to hear what is said in the room and 
> allows the people in the room to hear the WebEX participants.
> 
> This works for remote participates to follow what is 
> happening in the meeting room.  WebEX also allows remote 
> presentations.  The other does not.
>

---

From: "James M. Polk" <[email-masked]>
Date: Tue, 13 Apr 2010 21:32:28 -0500

At 04:58 PM 4/13/2010, Tony Li wrote:

>including half-way around the world.

having the same employer, I also use webex regularly - often with 
others half way around the world, but I've noticed something in my 
experience, and as others have said, it's mostly about the audio.

If I am on a call with 2, 5, 10, 50 other folks -- and they each are 
speaking into their mics -- no one complains of any problems.

If even one leg is into a conference room, especially if the talker 
is in that room, even worse: if the room has a conversation -- the 
audio goes to $hit for everyone else.

This is basically what we have at IETF meetings where there is one 
room that's a (large) conference room, and a bunch of individuals off 
on legs of the same webex.

I tried to get into the Hiroshima meetings, and it was a disaster, 
and I heard the same thing from other folks about Hiroshima and 
Anaheim, so it appears to be fairly consistent.

My guess -- is that whichever laptop was dialed into the webex didn't 
have the room audio (literally) plugged into the laptop; rather it 
was relying on the open inbound mic to feed the audio from the room 
speakers. This is fraught with engineering issues (reflections, echo, etc).

Conclusion:  Each room need to have an audio cable feeder from the 
speaker system within the room plugged directly into the laptop in 
that room's webex session.  This way, the only audio the webex 
connection gets is coming from the room speakers, and not the 
background noises from the speakers' mouths also.

James

>It works quite well and quite

---

From: Al Morton <[email-masked]>
Date: Tue, 13 Apr 2010 23:10:52 -0400

A couple of comments:

I think that webex can probably be made to work with room
audio systems, but it may take a level of audio mixing expertise
that can't be taught in a Sunday afternoon session, and it may
require more equipment than we have been able to commit to
the problem thus far. (I think this actually connects the 2 issues.)

Experience with webex fed into the room in Anaheim:
The room audio was a nightmare, on its own. We could hear conversations
from another room, or a radio station or something else at low level.
Audio level from webex was either too low to hear, or too distorted
to understand. There seemed to be a non-linearity in the level adjustment
(some sort of Automatic Gain Control in the transmission path?).

Alternate experiences: Two four-day meetings with a Skype conference
for remote participants. The main difference was that the host brought his
own audio mixing board to both meetings, and spent considerable time setting
levels before-hand and making adjustments as each participant called in.

Like others, I don't think that webex is at the root of the problem.
I do think that each new hotel/meeting room presents a complicated
set of problems for any remote participation method.
But we do need a solution to allow speakers to present slides
remotely. It may require some equipment purchases to accomplish this.

The 1-way audio stream and jabber have worked well for
all but the most active remote participants (and if you can reduce the
buffer time in your streaming audio client, you can avoid some of the
10 second delay, as Joel Jaeggli pointed-out during the Hiroshima meeting).

my 2 cents,
Al

---

From: Joel Jaeggli <[email-masked]>
Date: Tue, 13 Apr 2010 21:29:41 -0700

On 04/13/2010 07:32 PM, James M. Polk wrote:

> have the room audio (literally) plugged into the laptop; rather it was
> relying on the open inbound mic to feed the audio from the room

um... no. it was done with a pst-sip intertie to mixer in the room using
a digital inkeeper.

http://www.jkaudio.com/innkeeper1x.htm

> speaker system within the room plugged directly into the laptop in that
> room's webex session.  This way, the only audio the webex connection
> gets is coming from the room speakers, and not the background noises
> from the speakers' mouths also.

if you plug the input and the output into the same bus you get a loop.
it's fun.

---

From: Joel Jaeggli <[email-masked]>
Date: Tue, 13 Apr 2010 21:38:32 -0700

> from another room, or a radio station or something else at low level.

Most of the radio mikes provide by the hotel were packed in around uhf
channel 67, thanks to the fcc their equipment will finally be illegal by
june which is several years to late. wireless  shouldn't be used in that
sort of density and with that kind of overlap.

Several of the rooms on the 4th floor had the piped in musak audible on
their amplifiers even when turned down all the way.

One of the rooms on the 4th floor which was the cojoining of two smaller
rooms had an addible ground loop.

My expectations american hotel audio contractors are generally pretty
low but pasv in the hilton worked pretty hard to get in under them. If
you compare it to japan both the equipement and the people were on a
different level.

---

From: joel jaeggli <[email-masked]>
Date: Tue, 13 Apr 2010 23:08:47 -0700

> The Secretariat provides a laptop that runs the WebEX application.

actually no the webex application runs on whatever laptop is being used 
for the presentation typically that's the w.g. chair since it's also 
attached to the projector.

the audio is handled by a sip device dialed into the webex conference 
brdige number.

---

From: joel jaeggli <[email-masked]>
Date: Tue, 13 Apr 2010 23:12:28 -0700

On 4/13/2010 5:04 PM, [email-masked] wrote:
> Internally, we use to rely on WebEX or similar solutions for remote
> participation to a meeting. If all the participants are remote, no
> problem. Problems appear when remote participants are in the minority
> compared to the interested people in the meeting room. Floor control
> is especially quite challenging, even if the audio channel works
> well.

The is what could charitably be described as a bandwidth issue when one 
set of participants has a richer interaction than the other set, 
typically that's experienced as degridation by both parties to various 
degrees, this is part of the reason I'd rather take a conference call in 
the privacy of my own office.

> I share the idea that we could first ensure that slides are provided
> right in time before the meeting, ensure a good quality of the audio
> stream and encourage the use of the jabber room for exchange. About
> remote presentation, I'm afraid that the interactive part following
> the presentation will be too often time-consuming to be really
> efficient.
>
> Regards,
>
> Lionel
>
>> -----Message d'origine----- De : [email-masked]
>> Envoyé : mercredi 14 avril 2010 00:09 À : IETF WG Chairs Objet :
>> Re: WebEX in WG Sessions
>>
>>
>>> 0) I think the requirement is remote participation, not
>>> necessarily webex. webex is one tool to acheive this.
>>
>> It is complex because the sound system in the meeting room needs to
>> be connected to the WebEX session.  This allows the WebEX
>> participants to hear what is said in the room and allows the people
>> in the room to hear the WebEX participants.
>>
>> This works for remote participates to follow what is happening in
>> the meeting room.  WebEX also allows remote presentations.  The
>> other does not.
>>

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 14 Apr 2010 00:15:23 -0600

To add a bit to Joel's post....  Joel and others have tried all kinds of things to make this work but have not yet found something that makes the experience better than what you had. 

A large part of the problem that acoustic echo cancelation is hard - most algorithms work by basically make some approximate model of the room by using the signals in and out of the room. Consider a non IETF meeting in your average conference room that webex was designed to work with - perhaps a  situation with a speaker phone that has one speaker, one microphone or a speakerphone such that has a DSP that knows how to deal with multiple microphones, and assume a room size that is relatively small so the total time that an echo can exist is small. You can use a a simple model that assumes the signal is a combination of a small number of simple bounces with fairly limited delay. The search space to try various models parameters is constrained by length of delay, number of relevant reflections, and assumption of a linear model.  Probably most people on this email list have been on a webex call like this and had it work fine. If you did this from a phone with a reasonable ECAN (polycom starfish conference phone being one of many good examples), the phone's ecan probably took care of it. If you did it from a phone with inadequate ECAN, then the ecan on the webex conference bridges took care of it.

Now consider the IETF rooms. The output is computed from multiple speakers in different locations - the walls, speakers, and microphones are configured such that there can be multiple bounces and still have significant echo before hitting a microphone. There are multiple microphone that all get put on one channel before echo cancelation is attempted. Given the size of room, and multiple bounces, and speed of sound, you can get echo that are several hundred milliseconds or more. You have lots of ambient noise of computers and other people talking. This is way way outside the design parameters of any echo cancelation algorithm I have used. 

Cullen Jennings

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 14 Apr 2010 07:37:43 +0100

Russ

Here at Cisco, my training on using the Webex conference facilities was 
delivered using Webex itself in conference mode. That might be a way to 
deliver the training to those unable to attend a Sunday session.

- Stewart

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Wed, 14 Apr 2010 10:49:24 +0200

I am not sure whether it really changed in IETF 77 
but we had a really hard time to understand the 
remote WeBex users in the Hiroshima meeting and 
gave it up at the end.

Channeling jabber at mic is not really optimal.
I would like to use WeBex for remote presenters 
if it is a _real_ option for jabber+stream. 

If so, I would like to suggest to provide a
mobile phone number to ask for help immediately.
I think WeBex trainees are good but they will 
not be able to do what Joel was doing for us
(Thanks Joel!).

Cheers,
Mehmet

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of ext Cullen Jennings
> Sent: Tuesday, April 13, 2010 11:12 PM
> To: Russ Housley
> Cc: IETF WG Chairs
> 1) Obviously download the slides also works - but presenters 
> forget to say "on slide 7", so it is hard to know which 
> slides is currently displayed. I see some value in webex for 
> this this but having the jabber chat room indicate the slide 
> works about as well and removes some of the snafus of which 
> machine is being presented off of and webex "reformatting" slides
> 
> 2) The streaming media is sometimes easier to understand but 
> the latency is higher too. It might be nice to adjust the 
> coding to have less latency and very high audio quality.  
> 
> 3) This was the big value prop of using webex. In my mind it 
> has not worked and we should be careful about expanding webex 
> use until a solution is found that works. The issues of room 
> echo and echo cancelation have not yet been solved in the way 
> we are using webex. How we do floor control is not really 
> been figured out yet. Currently remote people that want to 
> participate have better luck just using the jabber channel 
> and having someone read their question at the mic. When that 
> beats your audio solution, you know the audio solution needs 
> some work. 
> 
> Overall if people have new ideas to make the remote audio 
> actually work, I'd like to see some experiments with theses 
> ideas but the current situation is not really solving 3. 
> 
> The remote presentation over webex have been a waste of WG 
> time and I would recommend people don't do that. In every 
> remote presentation I have seen, the presenting person could 
> not actually answer questions from the room due to the audio 
> being so messed up.
> 
> more inline ...
> 
>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Wed, 14 Apr 2010 11:13:34 +0200

One additional data point on the audio from Anaheim - we had several
remote participants for PWE3 (one-way, they didn't try to speak), and
at least one reported that the audio from Webex was extremely choppy
and hard to follow, and the Videolab audio streaming was much better
by comparison, so he listened to that instead. It's good that both
options were available. He was listening from the UK.

Cheers,
Andy

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 14 Apr 2010 08:16:25 -0700

On 04/14/2010 02:13 AM, Andrew G. Malis wrote:
> by comparison, so he listened to that instead.

The unidirectional audio stream confers a couple of benefit...

	zero echo cancelers in the path

	much greater dynamic range

	it starts as about an order of magnitude more data.

> It's good that both
>> A large part of the problem that acoustic echo cancelation is hard - most algorithms work by basically make some approximate model of the room by using the signals in and out of the room. Consider a non IETF meeting in your average conference room that webex was designed to work with - perhaps a  situation with a speaker phone that has one speaker, one microphone or a speakerphone such that has a DSP that knows how to deal with multiple microphones, and assume a room size that is relatively small so the total time that an echo can exist is small. You can use a a simple model that assumes the signal is a combination of a small number of simple bounces with fairly limited delay. The search space to try various models parameters is constrained by length of delay, number of relevant reflections, and assumption of a linear model.  Probably most people on this email list have been on a webex call like this and had it work fine. If you did this from a phone with a reasonable ECAN
 (polycom starfish conference phone being one of many good examples), the phone's ecan probably took care of it. If you did it from a phone with inadequate ECAN, then the ecan on the webex conference bridges took care of it.

---

From: joel jaeggli <[email-masked]>
Date: Wed, 14 Apr 2010 09:13:51 -0700

On 4/13/2010 1:31 PM, Carsten Bormann wrote:
> I'd feel much more comfortable to read this message after an
> s/WebEX/$PRODUCT/g. Are we really stuck with a mechanism that is so
> labor-intensive that it *requires* training (and consuming the
> attention of!) two facilitators per meeting?  Who is supposed to do
> this, if it detracts them from the meeting so much?  Clearly,
> facilitators are a good idea, but I have this nagging feeling that we
> are making a technological choice here that is a significant
> regression after all the wonderfulness we got out of vat/rat, wb, and
> nte in 1990s.

Well yeah there is a significant regession, we connected it to the pstn. 
  If you want to rebuild the mbone, be my guest.

If you remember back to vat/rat the net mutes mic button the 
applications generally did not operate in full duplex audio which 
substantially decreases the problem, e.g. there's no need echo 
cancellation anywhere in the system.

> Just a thought.
>
> But then, independent of remote participation, I'd love to have an
> audio watchdog at every meeting that physically forces people to
> properly attach/use their microphones, not pop into them, speak at an
> intelligible level and speed.  The chairs could use a slide advancer
> that knows how to advance both PPT and PDF (and a slide watchdog that
> makes sure these are readable and available in the MMM in the first
> place).  And a discussion-summarizer that makes sure important points
> are visible on the projectors.  And so on.  I'm not against
> facilitators, just mildly opposed to overloading their attention with
> another difficult-to-use tool.
>
> Gruesse, Carsten
>
>> Some WGs have been making use of WebEX to allow remote participants
>> to make presentations.  It also allows remote participants to
>> listen to the audio stream and see the slides that are being shown
>> in the room.  The popularity of the WebEX service has grown from
>> IETF76 to IETF77.  If this growth continues, then we will be
>> placing too great a burden on the volunteers that have been making
>> this service available.  I am considering a new requirement for a
>> WG to use WebEX, and I want to hear your thoughts before I take it
>> to the IESG.  Of course, IESG members are on this mail list, so
>> they will see this discussion, so they know it is coming.
>>
>> To reduce the burden on the WebEX volunteers, I am considering
>> asking each WG that wants to use WebEX during their meeting session
>> to have two people other than the WG Chairs trained on the WebEX
>> setup on Sunday. We have found that running the meeting and running
>> WebEX is too great a burden for the WG Chair; we need the WG Chair
>> to focus on the meeting, not the WebEX tools.   The trained people
>> will be responsible for starting the WebEX session, monitoring it
>> during the WG meeting, and then shutting it down at the end of the
>> WG meeting.  If problems are encountered, these trained people will
>> be able to call on the WebEX volunteers for help.  In this way, the
>> WebEX volunteers can focus on trouble spots.
>>
>> Thoughts?
>>
>> Russ
>

---

From: Behcet Sarikaya <[email-masked]>
Date: Wed, 14 Apr 2010 09:24:30 -0700 (PDT)

Hi Russ,
  Here is my 2 cents:
Wimax Forum uses Webex extensively. Webex is used only for presentations. Audio part is not used. Instead Intel bridge is used. 
I think that Wimax Forum teleconferences run very well and they can handle quite large number of people.
So my suggestion is to adopt something like this, which requires another donation to IETF J.
  Also I think that it is difficult to feed comments from Jabber remote participants into the meetings, I don't know how to solve this problem.
 
Regards,
 
Behcet

----- Original Message ----
> From: Russ Housley <[email-masked]>
> To: IETF WG Chairs <[email-masked]>
> Sent: Tue, April 13, 2010 5:09:02 PM
> necessarily

> right now

> seems complicated to setup.

It is complex because the sound system in the 
> meeting room needs to be
connected to the WebEX session.  This allows 
> the WebEX participants to
hear what is said in the room and allows the people 

> b) from what I've 
> heard before and agree, audio feed (maybe even one
> way), jabber and 
> slides sent in advance does a pretty good job. jabber
> relays for 
> questions.

This works for remote participates to follow what is happening 
> in the
meeting room.  WebEX also allows remote presentations.  The 

> based, that would require almost no training, much 

No.  The solution being used is VoIP 
> based.

Russ

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Apr 2010 16:50:08 -0400

--On Tuesday, April 13, 2010 16:09 -0400 Russ Housley
<[email-masked]> wrote:

> asking each WG that wants to use WebEX during their meeting
> session to have two people other than the WG Chairs trained on
> the WebEX setup on Sunday. We have found that running the
> meeting and running WebEX is too great a burden for the WG
> Chair; we need the WG Chair to focus on the meeting, not the
> WebEX tools.   The trained people will be responsible for
> starting the WebEX session, monitoring it during the WG
> meeting, and then shutting it down at the end of the WG
> meeting.  If problems are encountered, these trained people
> will be able to call on the WebEX volunteers for help.  In
> this way, the WebEX volunteers can focus on trouble spots.
> 
> Thoughts?

FWIW, I agree with your analysis.   If a WG can't have someone
other than chairs or jabber scribes managing the WebEX session,
and have that person reasonably well trained, then my experience
with WebEx is that it is Jabber and Audio feeds on steroids with
little value added to justify the additional complication.

   john

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 14 Apr 2010 12:58:42 -0400

>> this way, the WebEX volunteers can focus on trouble spots.

In some of the working groups I have been associated with, this is  
asking a lot.

If I am reading this right, a webex WG will effectively need

- 2 chairs
- a jabber scribe
- a regular scribe
- 2 "webex scribes"

i.e., 6 people, to make this work. I think that this means that

- some WG just shouldn't support webex or
- there will need to be paid staff support or
- there needs to be some volunteer carrot or stick to get people to  
sign up to do this.

As a concrete suggestion, maybe someone who takes the training and  
signs up for at least 2 sessions as a webex scribe should get  
registration at the student rate.

> little value added to justify the additional complication.

+1

Marshall

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 14 Apr 2010 10:04:52 -0700

>Wimax Forum uses Webex extensively. Webex is used only for presentations. Audio part is not used. Instead Intel bridge is used.

Is this in large meeting rooms like IETF meetings? I ask because the audio problems that we have seen in the last two meetings came mostly* from the acoustics of the large meeting rooms. If it is in large rooms, that is worth the IETF investigating.

*Some of the problems were from WebEx participants not muting their mics when someone else was talking, as we discovered in the IPsecME meeting in Anaheim. There were maybe five people logged in on WebEx, and at one point the person speaking became almost inaudible. After I barked "If you are not speaking, mute your mic *now*", a few clicks were heard and the audio got noticeably better immedately.

--Paul Hoffman, Director
--VPN Consortium

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 14 Apr 2010 18:33:24 +0100

Actually I don't wee why the second chair can't do the Webex.

When we used it for PWE3, and indeed when I set it up out of my Cisco 
a/c to use in TICTOC at IETF 77 we managed to share the chairing and the 
Webex management. The only assistance we had from the Webex support 
staff was to initially log in, and that was only because we did not have 
the a/c details.

As I recall we had one chair running the Webex control panel and the 
other loged in as a participant to display the slides and shared the 
various duties dynamically.

Stewart

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 14 Apr 2010 10:41:38 -0700

>Actually I don't wee why the second chair can't do the Webex.

There is not always a second chair. In the case of IPsecME, WebEx was used because the "second chair" was unable to attend the meeting. In a few other other WG meetings in Anaheim, one chair was running the meeting and the "second chair" was taking minutes and/or watching for Jabber comments.

--Paul Hoffman, Director
--VPN Consortium

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 14 Apr 2010 13:51:00 -0400

On Apr 14, 2010, at 1:33 PM, Stewart Bryant wrote:

> When we used it for PWE3, and indeed when I set it up out of my  
> Cisco a/c to use in TICTOC at IETF 77 we managed to share the  
> chairing and the Webex management. The only assistance we had from  
> the Webex support staff was to initially log in, and that was only  
> because we did not have the a/c details.
>

That will probably work ok as long as there is a second chair, and  
neither chair has any need to do anything else, like look up an old  
Internet draft or deal with jabber question.

Ten years ago, I remember some pretty complicated WG being run by 1  
chair with a pick-up scribe for meetings. We shouldn't explode the  
need for volunteers beyond the ability of the community to supply them.

Regards
Marshall

---

From: "Mike McBride (mmcbride)" <[email-masked]>
Date: Wed, 14 Apr 2010 10:55:37 -0700

Stig and I ran our wg using webex. It was not a burden and we don't need
webex assistance. My laptop was tied up for all of the presentations via
webex and Stig's was available. We had remote attendees thanking us for
providing them access to the wg meeting and said the audio quality was
great. 

We did not, however, have remote presenters. It would have been audio
painful. Remote participants could make comments on the webex chat. We
will continue to offer webex into our wg meetings but, due to the
bidirectional audio challenges into rooms, we will ask presenters to be
physically present.

mike

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Marshall Eubanks
> Sent: Wednesday, April 14, 2010 9:59 AM
> To: John C Klensin
> Cc: IETF WG Chairs; Russ Housley
> Subject: Re: [IAB] WebEX in WG Sessions
> 
> 
>

---

From: Fred Baker <[email-masked]>
Date: Wed, 14 Apr 2010 11:52:36 -0700

On Apr 14, 2010, at 10:55 AM, Mike McBride (mmcbride) wrote:

> We did not, however, have remote presenters. 

In my case, the issue driving the need for webex or something like it is remote presenters. Remote listening is supported by jabber, posted presentations, and audio. It's far from perfect, but it works. When I have a remote presenter I go looking for webex.

There is a part of me that would like to simply ban remote presenters. If you want to present at the meeting, attend the meeting. Most everyone else is there.

---

From: Marc Blanchet <[email-masked]>
Date: Wed, 14 Apr 2010 14:57:58 -0400

> There is a part of me that would like to simply ban remote presenters. If you want to present at the meeting, attend the meeting. Most everyone else is there.

I have the exact same opinion.

Marc.

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 14 Apr 2010 13:00:50 -0600

On 4/14/10 12:57 PM, Marc Blanchet wrote:
>> In my case, the issue driving the need for webex or something like it
>> is remote presenters. Remote listening is supported by jabber, posted
>> presentations, and audio. It's far from perfect, but it works. When I
>> have a remote presenter I go looking for webex.
>>
>> There is a part of me that would like to simply ban remote presenters.
>> If you want to present at the meeting, attend the meeting. Most
>> everyone else is there.
> 

Much as I like WebEx, I tend to agree.

Peter

---

From: joel jaeggli <[email-masked]>
Date: Wed, 14 Apr 2010 12:07:05 -0700

> I have the exact same opinion.

In opsec, we have a number of participants who cannot always attend, I 
have worked those folks in the past to insure that one of the document 
authors, contributors or the chairs is able to present the material up 
for discussion and more importantly is is prepared to run the discussion 
necessary to get the work done.

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 14 Apr 2010 15:38:37 -0400

If there were ever an organization that *ought* to be able to make 
remote participation in meetings work, it should be ietf. ISTM that it 
is worth continuing to try. But it seems we are not there yet for 
anything close to parity for local and remote attendees.

	Paul

---

From: Al Morton <[email-masked]>
Date: Wed, 14 Apr 2010 15:42:07 -0400

At 02:52 PM 4/14/2010, Fred Baker wrote:
>In my case, the issue driving the need for webex or something like 
>it is remote presenters. Remote listening is supported by jabber, 
>posted presentations, and audio. It's far from perfect, but it 
>works. When I have a remote presenter I go looking for webex.

+1, so far.

>There is a part of me that would like to simply ban remote 
>presenters. If you want to present at the meeting, attend the 
>meeting. Most everyone else is there.

In one of the WGs I chair, I've got an author who:
- is a true and practicing expert in the field
- we convinced to "take the pen" after floundering for a while
- has never attended an IETF meeting in person, AFAIK
- has one 20 minute presentation to do
- will probably only attend one session during IETF week
- and is located on another continent.

If discussion really gets going, I want him to be able to
participate in the most interactive way possible.
It would be satisfactory to make a pt-pt call with him,
as long as he could be heard clearly, and hear people speaking to him.

Otherwise, I get to present the slides, and can't replace his
knowledge when the questions are very detailed...

Al

---

From: "Bernard Aboba" <[email-masked]>
Date: Wed, 14 Apr 2010 12:43:44 -0700

Fred Baker said:

"In my case, the issue driving the need for webex or something like it is
remote presenters. Remote listening is supported by jabber, posted
presentations, and audio. It's far from perfect, but it works. When I have a
remote presenter I go looking for webex.

There is a part of me that would like to simply ban remote presenters. If
you want to present at the meeting, attend the meeting. Most everyone else
is there."

[BA] I attended IETF 76 remotely, and was surprised at how good the
experience was.  Between jabber, the meeting materials site and the
streaming audio (which was of very high quality), I had an excellent
experience, dramatically better than the last time I had attempted to attend
remotely, which was back in the days of the MBONE. 

Remote presentation is a trickier problem, as has been noted in other
postings.  I have seen a number of setups used to provide presenter audio,
including use of Skype with a variety of audio configurations (e.g. PC with
external speakers and/or headset, etc.).  I suspect we'll have to experiment
for a while before we figure out what works best for this.

---

From: "Mike McBride (mmcbride)" <[email-masked]>
Date: Wed, 14 Apr 2010 12:43:57 -0700

> have a remote presenter I go looking for webex.

It does work, but webex is better, sometimes much better. In my case,
the issue driving the need for webex, or something like it, is remote
attendees that don't have to guess what slide we are on and can
therefore easily follow the discussion. And when bidirectional audio
does work well they can participate verbally as if they are in the room.

> If you want to present at the meeting, attend the meeting. Most

Until we get the audio nailed down I agree. We did have a remote
presenter into a past wg meeting, using webex, and the interaction
worked flawless. But it was flawless because Joel and Gunter were all
over making the audio work great. We experienced how good remote mtg
access could be and still holding out hope for a repeat. 

We are the bloody IETF, we should be using cool, useful new technology.
A presenter shouldn't have to spend $2k in travel to present for 10
minutes especially if its coming out of their own pocket. Someday we'll
have telepresence units in each of the rooms and remote participants
skype (or whatever) into the meeting and their face pops up whenever
they are talking. IETF is rolling in the dough because the remote
attendance costs less and they attend in droves. Some day.

mike

---

From: joel jaeggli <[email-masked]>
Date: Wed, 14 Apr 2010 12:51:36 -0700

On 4/14/2010 12:38 PM, Paul Kyzivat wrote:
> remote participation in meetings work, it should be ietf.

we make remote participation work outside of the meetings, to the point 
where you can not attend them and still get your work done.

I've been working with or on various aspects of remote particpation at 
ietf meetings for something like 13 years, and I will say the assumption 
that we should be able to it better than other people is one of the 
impediments to it actually working.

The stuff that works reliably over and over again and scales well enough 
to be deployed by volunteers (important) are the mature 
least-common-demoninator things that everyone else uses. these days 
that's, radio mp3 streaming xmpp and to some extent sip dialout to 
conference bridges in the simple category.

> ISTM that it
>> for discussion and more importantly is is prepared to run the
>> discussion necessary to get the work done.
>>
>>
>>
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 14 Apr 2010 13:52:00 -0600

> anything close to parity for local and remote attendees.

Parity won't be achieved until we develop remote beering.

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 14 Apr 2010 16:11:34 -0400

On Apr 14, 2010, at 2:52 PM, Fred Baker wrote:

> presenters. If you want to present at the meeting, attend the meeting

Or make suitable arrangements (e.g., find a proxy).

Marshall

> . Most everyone else is there.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 14 Apr 2010 15:30:10 -0500

>> If you want to present at the meeting, attend the meeting
>

Just to agree with Marshall agreeing with Fred (and other +1s)...

It's hard to imagine that we need to work very hard to support remote 
presenters who can't find anyone in the room who's willing to support the 
topic being presented - my first guess is that the topic being presented 
isn't going anywhere.

The SIPPING co-chairs, and now the DISPATCH co-chairs, have been willing to 
put up one (or two?) slides on new ideas at the beginning of working group 
sessions, as a service to people who are looking for feedback, and I think 
that level of effort is very appropriate.

If a working group is willing to work seriously on a topic, most of the 
time, there should be someone who cares about it in the meeting room (if 
most of the people who care about it are remote participants, that's 
awesome, don't use up previous face-to-face slots, just have a 
teleconference that we DO know how to support very well :-)

Thanks,

Spencer

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 14 Apr 2010 13:38:32 -0700

At 12:43 PM -0700 4/14/10, Mike McBride (mmcbride) wrote:
>therefore easily follow the discussion.

Sorry, but that is just silly. The WG chair can enforce "have numbers on the slides, otherwise I won't allow them" *and* "say what slide number you are on". Similarly, I have noticed that Jabber scribes often say "slide 3", "slide 4" and so on so that remote people can follow along. These are infinitely more lightweight than WebEx, and already work today.

>And when bidirectional audio

That's quite true. I have not seen that happen with WebEx yet (data points are the very important TLS meeting in Hiroshima and the IPsecME meeting in Anaheim).

>access could be and still holding out hope for a repeat.

That is an unreasonable expectation; thus the original purpose of this thread.

>We are the bloody IETF, we should be using cool, useful new technology.

That is a non sequitur on many levels. First, the audio failures we are having is exactly because of the POTS interface we are using. Second, the IETF meeting theme is about getting things done, not showing off the latest technology. Third, we have recent experiences where the "cool, useful new technology" is taking away resources and providing some but little advantage; why should a volunteer organization that is not doing all that well financially ignore that?

>minutes especially if its coming out of their own pocket.

They don't have to. That's what the mailing list is for. WG chairs who have the problem of a valuable presenter who can't attend need to spend more time making the mailing list discussion work for them.

For IPsecME, WebEx for a remote presenter became a distraction that frustrated even the presenter. Others may have had different experiences, but I doubt we will ask for it again even if some presenters can't make it to Maastricht.

--Paul Hoffman, Director
--VPN Consortium

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 14 Apr 2010 22:44:57 +0200

On Apr 14, 2010, at 18:13, joel jaeggli wrote:

> pstn

I don't mind the PSTN.
(In fact, I'm sometimes moving the audio part of a teleconference to an Asterisk server in order to get reasonable intelligibility...)

Yes, the PTT aspect of vat was useful.

---

From: joel jaeggli <[email-masked]>
Date: Wed, 14 Apr 2010 14:07:27 -0700

On 4/14/2010 1:44 PM, Carsten Bormann wrote:
> I don't mind the PSTN. (In fact, I'm sometimes moving the audio part
> of a teleconference to an Asterisk server in order to get reasonable
> intelligibility...)

well, if we were  to aspire 100% end-to-end sip we could probably step 
up to some wideband codec beyond just pcm, that would give us more to 
work with, it would probably drop several noise-gates and echo cancelers 
from the path in the process.

> Yes, the PTT aspect of vat was useful.

it's useful when working the HF band as well.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 14 Apr 2010 23:15:22 +0200

> Yes, the PTT aspect of vat was useful.

(I hate it when Apple Mail decides to ship incomplete messages.)

vat also had this wonderful per-participant set of mute buttons you could use to silence Mr clickety-click...

What I do not like about the PSTN is:
-- the 4 kHz limitation,
-- the incredibly bad technology that is being used at the end ($5 speakerphones),
-- the expectation of a "conference bridge" (= small room) experience.

Something like mumble/murmur may be much closer on the audio side to what we need than WebEX.
Or actually, if somebody external wants to say something on the mike, let them record it, and have it queued up for playback at the right time by a chair.
For remote presentation, use VNC for slide control and some higher fidelity audio technology (Skype, iChat, at least G.722 SIP which lots of $40 SIP phones now have).
[I once gave a talk using iChat to a large room that was quite well received [talking about the presentation quality :-]].

With the possible exception of VNC (which only CS people can use :-), this is actually all mass market technology.

Gruesse, Carsten

---

From: "Mike McBride (mmcbride)" <[email-masked]>
Date: Wed, 14 Apr 2010 14:27:16 -0700

I am definitely one silly guy. And I have a wg which is happy that we
are providing this type of remote access. When they are happy, I am
happy. And I will use the next cool, useful, new (and donated)
technology that comes along if it provides an even better experience for
them. 

mike

> From: Paul Hoffman [mailto:[email-masked]]
> Sent: Wednesday, April 14, 2010 1:39 PM
> To: Mike McBride (mmcbride); Fred Baker (fred)
> Cc: IETF WG Chairs; Russ Housley; John C Klensin
> Subject: RE: [IAB] WebEX in WG Sessions
> 
> Sorry, but that is just silly. The WG chair can enforce "have numbers
> on the slides, otherwise I won't allow them" *and* "say what slide
> number you are on". Similarly, I have noticed that Jabber scribes
often
> say "slide 3", "slide 4" and so on so that remote people can follow
> along. These are infinitely more lightweight than WebEx, and already
> work today.
> 
> That's quite true. I have not seen that happen with WebEx yet (data
> points are the very important TLS meeting in Hiroshima and the IPsecME
> meeting in Anaheim).
> 
> presenters.
> That is a non sequitur on many levels. First, the audio failures we
are
> having is exactly because of the POTS interface we are using. Second,
> the IETF meeting theme is about getting things done, not showing off
> the latest technology. Third, we have recent experiences where the
> "cool, useful new technology" is taking away resources and providing
> some but little advantage; why should a volunteer organization that is
> not doing all that well financially ignore that?
> 
> They don't have to. That's what the mailing list is for. WG chairs who
> have the problem of a valuable presenter who can't attend need to
spend
> more time making the mailing list discussion work for them.
> 
> For IPsecME, WebEx for a remote presenter became a distraction that
> frustrated even the presenter. Others may have had different
> experiences, but I doubt we will ask for it again even if some
> presenters can't make it to Maastricht.
> 
> --Paul Hoffman, Director
> --VPN Consortium

---

From: Christian Vogt <[email-masked]>
Date: Thu, 15 Apr 2010 10:05:43 +0200

Russ and all:

I agree that someone other than the WG chairs should be responsible for WebEX, just like we already use dedicated minute takers and Jabber scribes.  It is true that finding volunteers is difficult.  But this is a non-issue in my personal opinion given that WebEX is optional for WG sessions at IETF meetings.  (FWIW, the same does not apply for WG interim meetings, where WebEX is typically the main communication method.)  This said, one WebEX volunteer may be enough.  It is important to have two volunteers for meeting minutes, but I don't see this requirement for WebEX.

- Christian

---

From: "Elwell, John" <[email-masked]>
Date: Thu, 15 Apr 2010 11:10:40 +0200

Folks are paying good money to attend the IETF in person, so whatever we do should not detract from the experience of those paying their dues. Consequently we must avoid any disruption to the meeting, e.g.,  through delays in getting any tools set up and through interference with audio in the room from badly disciplined remote participants.

Also, whilst we have to do some things to support remote participation, we have to draw the line somewhere, and not make remote participation so attractive that a large part of the community relies exclusively on remote participation, thereby avoiding fees and pushing up the fees for those still attending in person (unless it is our declared policy to migrate towards virtual meetings). So in the exceptional case where it is for the benefit of a WG to have a particular topic presented remotely, I agree we need a solution, and I agree it would be good to relieve the chairs of any burden involved in setting up any tools involved. But for now, keep that as the exception rather than the rule.

By the way, I have participated remotely only once (during the Stockholm IETF), and I found the combination of audio streaming and jabber reasonably sufficient, certainly better than nothing.

John

> [mailto:[email-masked]] On Behalf Of Fred Baker
> Sent: 14 April 2010 19:53
> To: Mike McBride
> In my case, the issue driving the need for webex or something 
> like it is remote presenters. Remote listening is supported 
> by jabber, posted presentations, and audio. It's far from 
> perfect, but it works. When I have a remote presenter I go 
> looking for webex.
>

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Thu, 15 Apr 2010 11:11:58 +0200

As a WG chair I would note I am a volunteer. I don't have to do that job.

Any use of Webex to support remote participants is going to involve WG chairs in extra work, even if they do not have to run the Webex themselves.

I certainly will support that usage only if it makes my work easier in other directions.

Thus for example, if the only reason for use is to support the presentation of "yet another author draft that the WG currently does not need on its schedule", then I am unlikely to try and organise something for someone who should essentially be organising themselves (by attending the meeting themselves or getting a proxy). Particularly if the person not attending is failing to make any financial contribution at all to the work of the IETF.

Therefore I hope we are not turning this into a "right" for remote participants, but rather something at the WG chair discretion.

regards

Keith 

> [mailto:[email-masked]] On Behalf Of Spencer Dawkins
> Sent: Wednesday, April 14, 2010 9:30 PM
> To: Marshall Eubanks; Fred Baker
> Cc: IETF WG Chairs; John C Klensin
> It's hard to imagine that we need to work very hard to 
> support remote presenters who can't find anyone in the room 
> who's willing to support the topic being presented - my first 
> guess is that the topic being presented isn't going anywhere.
> 
> The SIPPING co-chairs, and now the DISPATCH co-chairs, have 
> been willing to put up one (or two?) slides on new ideas at 
> the beginning of working group sessions, as a service to 
> people who are looking for feedback, and I think that level 
> of effort is very appropriate.
> 
> If a working group is willing to work seriously on a topic, 
> most of the time, there should be someone who cares about it 
> in the meeting room (if most of the people who care about it 
> are remote participants, that's awesome, don't use up 
> previous face-to-face slots, just have a teleconference that 
> we DO know how to support very well :-)
> 
> Thanks,
> 
> Spencer 
> 
>

---

From: Eliot Lear <[email-masked]>
Date: Thu, 15 Apr 2010 15:34:01 +0200

Paul,

On 4/14/10 10:38 PM, Paul Hoffman wrote:
> That is a non sequitur on many levels. First, the audio failures we are having is exactly because of the POTS interface we are using.

I think Cullen has pointed out that it's not just POTS, but the state of 
the art with regard to echo cancellation in large spaces.

> Second, the IETF meeting theme is about getting things done, not showing off the latest technology.

Let's put it in other terms: wouldn't it be nice if we could use this 
sort of technology so that more people could participate in a more 
interactive fashion than otherwise would?  My own favorite example is 
the current meltdown that is going on in the RRG, due at least in some 
part because there's one person who can never and will never get funding 
to travel, and so he has 0 human interaction with the people involved.  
It'd be nice if he could come, but having something like WebEx might 
have at least kept things from deteriorating.

> Third, we have recent experiences where the "cool, useful new technology" is taking away resources and providing some but little advantage; why should a volunteer organization that is not doing all that well financially ignore that?

I'm sorry- what resources are you talking about?
> They don't have to. That's what the mailing list is for. WG chairs who have the problem of a valuable presenter who can't attend need to spend more time making the mailing list discussion work for…

Mailing lists are no substitute for some sort of interaction.  Otherwise 
we wouldn't have face to face meetings.  It's not clear that WebEx is 
either, by the way.  It takes a different skill set to present over 
WebEx, in my experience, by the way.

On the other hand, my own experience is split on this: as you heard in 
the plenary the last time, there was one working group that met F2F 
where the participants flew in and flew out, and to me that wasn't 
sufficient time to actually have discussion with the individuals, and 
that IMHO the group has suffered as a result.  A WebEx fly-in could be 
worse in those situations.

> For IPsecME, WebEx for a remote presenter became a distraction that frustrated even the presenter. Others may have had different experiences, but I doubt we will ask for it again even if some prese…

Yes, and WebEx was the first time we had a disruptive individual in 
attendance?  The nice thing about WebEx is that you can shut such a 
person up.  Try that in person.

Eliot

---

From: Ted Lemon <[email-masked]>
Date: Thu, 15 Apr 2010 10:33:10 -0400

On Apr 15, 2010, at 5:10 AM, Elwell, John wrote:
> Also, whilst we have to do some things to support remote participation, we have to draw the line somewhere, and not make remote participation so attractive that a large part of the community relies exclusively on remote participation, thereby avoiding fees and pushing up the fees for those still attending in person (unless it is our declared policy to migrate towards virtual meetings).

The underlying theme here is that remote participants don't add value.   However, this is incorrect, at least in my experience--although some of us (me included) have the luxury of attending IETF on our employer's dime, and it's part of our job description, others of us have to justify the expenditure, and there are those whose companies simply can't afford to send them halfway around the world every four months.

It was the case at the most recent IETF that two participants in the dhc working group who are working on a draft that needed to be discussed couldn't attend.   So that work didn't get discussed.   I suspect this happens a lot.   We haven't historically done WebEx in the dhc working group, and didn't try this time, but the point is that remote participation can add value, and I think it's incorrect to privilege fee-paying visitors over other active participants.

Also, I get the impression that it's fairly routine for a certain (albeit small) number of active participants to be denied visas for whatever country IETF happens to be in; those participants obviously did not choose not to attend, and hence shouldn't be penalized.

---

From: Behcet Sarikaya <[email-masked]>
Date: Thu, 15 Apr 2010 08:26:27 -0700 (PDT)

----- Original Message ----
> From: Paul Hoffman <[email-masked]>
> To: Behcet Sarikaya <[email-masked]>; Russ Housley <[email-masked]>; IETF WG Chairs <[email-masked]>
> Sent: Wed, April 14, 2010 12:04:52 PM
>Wimax Forum uses Webex 
> extensively. Webex is used only for presentations. Audio part is not used. 
> Instead Intel bridge is used.

Is this in large meeting rooms like IETF 
> meetings? I ask because the audio problems that we have seen in the last two 
> meetings came mostly* from the acoustics of the large meeting rooms. If it is in 
> large rooms, that is worth the IETF investigating.

*Some of the problems 
> were from WebEx participants not muting their mics when someone else was 
> talking, as we discovered in the IPsecME meeting in Anaheim. There were maybe 
> five people logged in on WebEx, and at one point the person speaking became 
> almost inaudible. After I barked "If you are not speaking, mute your mic *now*", 
> a few clicks were heard and the audio got noticeably better 
> immedately.

I don't know about meeting room situation, I always attend the meetings from home :-).

--b

---

From: Ed Juskevicius <[email-masked]>
Date: Thu, 15 Apr 2010 11:31:43 -0400

I've been watching this discussion, and reflecting on my (limited)
experiences with Webex from my IAOC days.  It was certainly true that we
often had to fiddle for more time at the start of our meetings (eg. 5
minutes) to get Webex running, compared to just using audioconferencing.
That was an annoyance for me, especially when our meetings were scheduled to
only last one hour.

On the other hand, my most recent experience wsa fine, but it did not
involving any large room with poor acoustics.  It also may have helped that
someone who knew what he was doing ran the webex session.  All I had to do
was be available to receive a webex call out to me at the right time.

I think we need more running code with webex in an IETF setting to progress
this discussion.  To facilitate this, you may want to arrange training for a
Sunday afternoon session in July along the lines of "How to use Webex in an
IETF meeting room (with poor acoustics and lots of microphones) with the
least amount of pain".  Things to watch for should include:

1) The number of people who attend the tutorial; and
2) How many (more) WGs try using webex during IETF 78.

Regards,

Ed Juskevicius
[email-masked]

On Thu, Apr 15, 2010 at 10:33 AM, Ted Lemon <[email-masked]> wrote:

> > Folks are paying good money to attend the IETF in person, so whatever we
> do should not detract from the experience of those paying their dues.
> Consequently we must avoid any disruption to the meeting, e.g.,  through
> delays in getting any tools set up and through interference with audio in
> the room from badly disciplined remote participants.
> >
> > Also, whilst we have to do some things to support remote participation,
> we have to draw the line somewhere, and not make remote participation so
> attractive that a large part of the community relies exclusively on remote
> participation, thereby avoiding fees and pushing up the fees for those still
> attending in person (unless it is our declared policy to migrate towards
> virtual meetings).
>
> The underlying theme here is that remote participants don't add value.
> However, this is incorrect, at least in my experience--although some of us
> (me included) have the luxury of attending IETF on our employer's dime, and
> it's part of our job description, others of us have to justify the
> expenditure, and there are those whose companies simply can't afford to send
> them halfway around the world every four months.
>
> It was the case at the most recent IETF that two participants in the dhc
> working group who are working on a draft that needed to be discussed
> couldn't attend.   So that work didn't get discussed.   I suspect this
> happens a lot.   We haven't historically done WebEx in the dhc working
> group, and didn't try this time, but the point is that remote participation
> can add value, and I think it's incorrect to privilege fee-paying visitors
> over other active participants.
>
> Also, I get the impression that it's fairly routine for a certain (albeit
> small) number of active participants to be denied visas for whatever country
> IETF happens to be in; those participants obviously did not choose not to
> attend, and hence shouldn't be penalized.
>
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 15 Apr 2010 11:23:47 -0500

In addition to Ed's helpful observations and suggestions, I'd also recommend 
that anyone who has opinions about this topic also subscribe to 
https://www.ietf.org/mailman/listinfo/vmeet, which is actually the mailing 
list that was set up to "Explore, specify and develop improved services for 
remotely participating in IETF meetings".

That would also have the benefit of interacting with people who are smart 
about this topic and are NOT working group chairs/IESG/IAB...

Thanks,

Spencer

----- Original Message ----- 
From: Ed Juskevicius
To: Russ Housley
Cc: [email-masked]
Sent: Thursday, April 15, 2010 10:31 AM
Subject: Re: [IAB] WebEX in WG Sessions

Regards,

Ed Juskevicius
[email-masked]

---

From: Joel Jaeggli <[email-masked]>
Date: Thu, 15 Apr 2010 20:53:37 -0700

> The underlying theme here is that remote participants don't add

I don't think that's an underlying theme, I think there are a number of
compromises that need to be made deliberately and discussing them
helps... I haven't worked for something like 10 years on ietf remote
participation activities because I thought remote participants didn't
add value.

>  However, this is incorrect, at least in my
> experience--although some of us (me included) have the luxury of
> attending IETF on our employer's dime, and it's part of our job
> description, others of us have to justify the expenditure, and there
> are those whose companies simply can't afford to send them halfway
> around the world every four months.

On the other hand some of use have paid to attend ietf meeting out of
our own pockets because we believed that the activity were were engaged
in was sufficiently important.  If the threshhold is you have to be
there to participate, in order to advance the work, either it's
important enough to attend or maybe the work doesn't get done, that happens.

> It was the case at the most recent IETF that two participants in the
> dhc working group who are working on a draft that needed to be
> discussed couldn't attend.   So that work didn't get discussed.   I
> suspect this happens a lot.   We haven't historically done WebEx in
> the dhc working group, and didn't try this time, but the point is
> that remote participation can add value, and I think it's incorrect
> to privilege fee-paying visitors over other active participants.
> 
> Also, I get the impression that it's fairly routine for a certain
> (albeit small) number of active participants to be denied visas for
> whatever country IETF happens to be in; those participants obviously
> did not choose not to attend, and hence shouldn't be penalized.
>

---

From: Ted Lemon <[email-masked]>
Date: Fri, 16 Apr 2010 07:29:50 -0400

On Apr 15, 2010, at 11:53 PM, Joel Jaeggli <[email-masked]> wrote:

> helps...

By "here" I meant the specific message to which I was replying, not the IETF or IAB as a whole, nor even the person who wrote the original message.

But as for paying out of pocket to attend, this costs more than many people make in four months after taxes and basic expenses, so I don't think it's fair to say that a person's failure to attend in person can be counted as an indication that their work is not important enough.

---

From: "David Harrington" <[email-masked]>
Date: Fri, 16 Apr 2010 09:48:38 -0400

Hi,

Maybe what we need to do is tailor our 'attendance" fees.

We have been experimenting with day fees, to accommodate those who do
not wish to attend all week. 
Maybe we should have a "remote attendance fee", for those who, for
whatever reason, want to attend remotely. Users could be required to
register and pay for the (Webex?) session, and use their registration
ID to gain access. That would allow us to hire people to coordinate
and troubleshoot the remote attendance tools (audio, wbex, and jabber
offerings) for remote participation. 

I prefer that people attend the meetings in person so they interact
face to face with others who might be interested in the topic, even
when the topic might not be quite appropriate for the session in which
the presentation is given. Live attendance would also allow for the
person to gain a better understanding of what else is happening
(similar to the debate over day attendance).

The cost of remote support should probably be included in the
weekly/daily attendance fees, since many attendees utilize the same
services during a meeting. Just as we do not restrict the published
proceedings to those who attended, the audio/jabber/webex archives
should not be restricted to those who paid for live remote access.

Sometimes remote particpation is the only option, or is a preferred
option for various reasons. If we are going to offer it, let's try to
offer the best way we can, determine the costs associated with the
service, and charge those who use that service accordingly. 

dbh

> [mailto:[email-masked]] On Behalf Of Joel Jaeggli
> Sent: Thursday, April 15, 2010 11:54 PM
> To: Ted Lemon
> Cc: Russ Housley; IETF WG Chairs

---

From: Ed Juskevicius <[email-masked]>
Date: Fri, 16 Apr 2010 10:34:31 -0400

I like your idea.  My first reaction is to say:

   1. Remote participation (largely one-way, listen only) via audiostreaming
   and/or jabber should continue to be offered to the community, at the current
   price (i.e. $0); and
   2. "Better QoS" remote participation (utilizing webex, run by people who
   are paid to adjust the audio quality during a wg session) sounds like a good
   idea if we think  enough people would pay to remotely participate in
   IETF using webex.

But, could the IETF afford # 2?

Anyone who pays a fee to participate remotely via webex should have every
right to expect a QoS for that experience that is worth the money.  Also, if
the QoS is any good, then some fraction of current IETF attendees (say 10%)
might stay home versus going to a meeting.

I wonder what a for-profit entity would charge the IETF to manage a webex
environment for an entire IETF meeting, with mixing boards and people in 8
rooms at a time.  It would require travel and living for 8 people for more
than a week ... Say $3K x 8 = $24K.  In addition, a for-profit
organization would charge for labor, other overhead plus some margin ... Say
$500 per day per person x 8 people x 9 days = $36K.

$24K + $36K = $60K.

I wonder if people would be willing to pay $600 for a webex password to
access all IETF sessions during a week of meetings?

If yes, we'd need 100 NEW people to sign-up for webex to pay the costs of
the arrangement, plus every existing attendee who chooses to stay home
rather than travel would need to sign-up for this too.

This means we'd probably need between 150 and 200 people to sign-up for
webex AND to pay $600 each for it to be sustainable in a long run scenario.

Regards,

Ed Juskevicius

---

From: Joel Jaeggli <[email-masked]>
Date: Fri, 16 Apr 2010 07:34:29 -0700

one thing that a remote attendance fee implies is better than simple
best effort service provided by volunteers and paid staff with other
responsibilities.

if you figure that present and including the audio streaming that the
concurrent (and realtime) remote audience floats at around 10%-15% of
the size of the attendee population in the rooms, it's not a large base
to build such a service on and you can expect some attrition when the
pay-wall goes up.

joel

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 16 Apr 2010 10:41:53 -0400

I'm skeptical of, but not offended by, the "remote attendance fee" idea.  If people voted with their wallets in sufficient numbers, that would fix my skepticism.  As others have pointed out, if folks are paying for remote attendance, then the bar is much higher for the quality that must be provided, and harking further back in the thread, it's not reasonable to expect this to be provided by the WG chairs or volunteers.  So costs could be expected to be high.

I *am* opposed to (and surprised by) the notion that the in-person fee should be increased to pay for such an experiment.  

--John

---

From: Marshall Eubanks <[email-masked]>
Date: Fri, 16 Apr 2010 10:53:17 -0400

On Apr 16, 2010, at 10:41 AM, John G. Scudder wrote:

> I'm skeptical of, but not offended by, the "remote attendance fee"  
> idea.  If people voted with their wallets in sufficient numbers,  
> that would fix my skepticism.  As others have pointed out, if folks  
> are paying for remote attendance, then the bar is much higher for  
> the quality that must be provided, and harking further back in the  
> thread, it's not reasonable to expect this to be provided by the WG  
> chairs or volunteers.  So costs could be expected to be high.

I am not wearing any hats here, and this is just my personal opinion,  
but I  worry more about the virtual interim. We are clearly seeing  
more of them, and quality is essential there, as all presentations are  
remote. If they are
valuable, they probably need to be supported and that would cost  
money. Would people pay a registration fee to attend a virtual  
interim ? Or, should we be looking for sponsors for that ?

>
> I *am* opposed to (and surprised by) the notion that the in-person  
> fee should be increased to pay for such an experiment.

At present, the IETF has two main funding sources -

- sponsorship (including ISOC)
- in-person registration fees and related income sources (mainly,  
hotel commissions from hotel room spending)

If costs go up, and nothing else is cut, the money has to come from  
one of these sources, or a new one has to be created...

Regards
Marshall

---

From: Joel Jaeggli <[email-masked]>
Date: Fri, 16 Apr 2010 08:00:24 -0700

On 04/16/2010 07:53 AM, Marshall Eubanks wrote:
>> idea.  If people voted with their wallets in sufficient numbers, that
>> would fix my skepticism.  As others have pointed out, if folks are
>> paying for remote attendance, then the bar is much higher for the
>> quality that must be provided, and harking further back in the thread,
>> it's not reasonable to expect this to be provided by the WG chairs or
>> volunteers.  So costs could be expected to be high.
> 
> but I  worry more about the virtual interim. We are clearly seeing more
> of them, and quality is essential there, as all presentations are
> remote. If they are
> valuable, they probably need to be supported and that would cost money.
> Would people pay a registration fee to attend a virtual interim ? Or,
> should we be looking for sponsors for that ?

>From the perspective of a wg chair, a purely virtual interim has lot
fewer thing going on in the cockpit, a lot of which people do on a
fairly routine basis anyway.

> 
>>
>> I *am* opposed to (and surprised by) the notion that the in-person fee
>> should be increased to pay for such an experiment.
> 
> - in-person registration fees and related income sources (mainly, hotel
> commissions from hotel room spending)
> 
> If costs go up, and nothing else is cut, the money has to come from one
> of these sources, or a new one has to be created...
> 
> Regards
> Marshall
> 
> 
> 
>>
>>
>> --John
>>

---

From: Ed Juskevicius <[email-masked]>
Date: Fri, 16 Apr 2010 11:08:10 -0400

+1.

I should have clarified that I liked David's idea of thinking about creating
a new fee for "good" webex remote participation.

I do *not* support the idea of increasing everyone else's meeting fee to
partially offset any new costs for remote participation.

Regards,

Ed

On Fri, Apr 16, 2010 at 10:41 AM, John G. Scudder <[email-masked]> wrote:

> I'm skeptical of, but not offended by, the "remote attendance fee" idea.
>  If people voted with their wallets in sufficient numbers, that would fix my
> skepticism.  As others have pointed out, if folks are paying for remote
> attendance, then the bar is much higher for the quality that must be
> provided, and harking further back in the thread, it's not reasonable to
> expect this to be provided by the WG chairs or volunteers.  So costs could
> be expected to be high.
>

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 16 Apr 2010 11:12:34 -0400

On Apr 16, 2010, at 10:53 AM, Marshall Eubanks wrote:
> If costs go up, and nothing else is cut, the money has to come from one of these sources, or a new one has to be created...

Understood.  I thought David's proposal was that the effort be self-funded through a new fee-paying category of remote attendee.  I'm OK with that although as I noted, skeptical that there would be sufficient uptake to bring in the needed funds (in which case I'd expect the experiment to be shut down).

I guess it's unreasonable to ask such an experiment to be self-funding ab initio, so other funds would probably have to be sought for the first iteration were it to be tried.  As you might guess I'd frown on using IETF general funds (or whatever they're called) for this purpose, but it would seem OK to seek a sponsor for it.

--John

---

From: "David Harrington" <[email-masked]>
Date: Fri, 16 Apr 2010 11:18:25 -0400

Hi,

I realized after I sent it that my wording was probably poor.
I was not proposing that normal fees be increased to support the webex
option.
What I meant to say was that people who paid for live attendance
should not be assessed any additional fees to utilize the webex
service (if, for example, they chose to "remote attend" the 9am
session from their hotel room, or "remote attend" a session while live
attending another).

dbh

> I *am* opposed to (and surprised by) the notion that the 
> in-person fee should be increased to pay for such an experiment.  
> 
> --John
> 
> >> On the other hand some of use have paid to attend ietf meeting
out
> > of

---

From: "David Harrington" <[email-masked]>
Date: Fri, 16 Apr 2010 11:19:54 -0400

Hi,
 
Ed, thanks for doing the quick financial analysis. That gives a bit
better info to work with.
It sounds expensive described your way.
But I note that the estimated $600 cost for access to all IETF
sessions via Webex is less than the normal IETF weekly fee to all IETF
sessions.
 
It could be dramatically cheaper for a company to participate via
Webex than paying airfare and hotel costs.
And to cover all eight "tracks" live would require a company to send
eight people (about $24k by your numbers)
So, in comparison, $4800 (8x$600) is pretty cheap. 
 
I agree the SLA could be problematic, 
If I understand correctly, current users of webex support aren't
paying the normal IETF fees.
This Webex-participation could be charged as the normal IETF fee, 
Live attendees are not guarateed any quality of service (and some
sessions would certainly fail to meet expectations)
Attendees could choose whether to register to attend live or to use
the Webex support, and then the collected fees could be allocated
accordingly.
 
The terms and conditions for webex support could specifiy that the
quality is not guaranteed (and no cookies are provided).
Some of the quality issues might be solvable by using paid staff
rather than (our excellent) volunteers.
Some QoS issues are just technology-related (echo suppression, etc.)
and cannot be solved easily.
 
dbh

  _____  

From: Ed Juskevicius [mailto:[email-masked]] 
Sent: Friday, April 16, 2010 10:35 AM
To: David Harrington
Cc: IETF WG Chairs; Russ Housley
Subject: Re: [IAB] WebEX in WG Sessions

1.	Remote participation (largely one-way, listen only) via
audiostreaming and/or jabber should continue to be offered to the
community, at the current price (i.e. $0); and 

2.	"Better QoS" remote participation (utilizing webex, run by
people who are paid to adjust the audio quality during a wg session)
sounds like a good idea if we think  enough people would pay to
remotely participate in IETF using webex. 

Anyone who pays a fee to participate remotely via webex should have
every right to expect a QoS for that experience that is worth the
money.  Also, if the QoS is any good, then some fraction of current
IETF attendees (say 10%) might stay home versus going to a meeting.
 
I wonder what a for-profit entity would charge the IETF to manage a
webex environment for an entire IETF meeting, with mixing boards and
people in 8 rooms at a time.  It would require travel and living for 8
people for more than a week ... Say $3K x 8 = $24K.  In addition, a
for-profit organization would charge for labor, other overhead plus
some margin ... Say $500 per day per person x 8 people x 9 days =
$36K.  
 
$24K + $36K = $60K.
 
I wonder if people would be willing to pay $600 for a webex password
to access all IETF sessions during a week of meetings?
 
If yes, we'd need 100 NEW people to sign-up for webex to pay the costs
of the arrangement, plus every existing attendee who chooses to stay
home rather than travel would need to sign-up for this too.  
 
This means we'd probably need between 150 and 200 people to sign-up
for webex AND to pay $600 each for it to be sustainable in a long run
scenario.

Regards,
 
Ed Juskevicius  

On Fri, Apr 16, 2010 at 9:48 AM, David Harrington
<[email-masked]> wrote:

Hi,

dbh

---

From: "David Harrington" <[email-masked]>
Date: Fri, 16 Apr 2010 11:45:28 -0400

Hi,

I think it might be able to be self-funding if access to both the live
audio and webex support required payment of the normal weekly IETF
fees. 

Many people are utilizing these services and paying nothing for them.
Why should they get them for free? Other people are paying meeting
fees to attend the same sessions. 

Why should our volunteers continue to provide these valuable services
for free? They should be able to be paid for this obviously valuable
service (if they want to bid for the contract, etc.)

If people want these valuable services, then let's make it a feature
of the IETF meeting fee - for those who are already paying the meeting
fees, they'll continue to get this extra service for no extra charge.
For those who are NOT paying the IETF meeting fees, they will need to
start paying the meeting fees to get remote access to the same
meetings. With the funds collected for remote access, we can then
support staff to manage live audio and webex support. 

I would even consider adding a delay to the free access to the audio
and webex archives for those who have not paid meeting fees. Make the
free archives available only when the proceedings are published. If
you want immediate access (live audio or webex) or access to archives
before the proceedings are published, then payment of IETF meeting
fees are required. But delaying free archive access is a separate
discussion.

dbh

> From: John G. Scudder [mailto:[email-masked]] 
> Sent: Friday, April 16, 2010 11:13 AM
> To: Marshall Eubanks
> Cc: David Harrington; IETF WG Chairs
> > If costs go up, and nothing else is cut, the money has to 
> come from one of these sources, or a new one has to be created...
> 
> Understood.  I thought David's proposal was that the effort 
> be self-funded through a new fee-paying category of remote 
> attendee.  I'm OK with that although as I noted, skeptical 
> that there would be sufficient uptake to bring in the needed 
> funds (in which case I'd expect the experiment to be shut down).
> 
> I guess it's unreasonable to ask such an experiment to be 
> self-funding ab initio, so other funds would probably have to 
> be sought for the first iteration were it to be tried.  As 
> you might guess I'd frown on using IETF general funds (or 
> whatever they're called) for this purpose, but it would seem 
> OK to seek a sponsor for it.
> 
> --John
>

---

From: Yaron Sheffer <[email-masked]>
Date: Fri, 16 Apr 2010 18:51:54 +0300

Hi Marshall,

viewed purely from the technical (audio) perspective, our experience at
IPsecME has been that virtual interims are a simple thing that can be
managed effectively, at zero cost, using "good enough" open source tools
(see http://trac.tools.ietf.org/wg/ipsecme/trac/wiki/ConferenceCalls).

Remote participation at face-to-face sessions is a whole different
matter, as was pointed out by many on this thread.

Thanks,
	Yaron

---

From: John C Klensin <[email-masked]>
Date: Fri, 16 Apr 2010 11:55:20 -0400

--On Friday, April 16, 2010 10:53 -0400 Marshall Eubanks
<[email-masked]> wrote:

>> I'm skeptical of, but not offended by, the "remote attendance
>> fee"   idea.  If people voted with their wallets in
>> sufficient numbers,   that would fix my skepticism.  As
>> others have pointed out, if folks   are paying for remote
>> attendance, then the bar is much higher for   the quality
>> that must be provided, and harking further back in the  
>> thread, it's not reasonable to expect this to be provided by
>> the WG   chairs or volunteers.  So costs could be expected to
>> be high.
> 
> I am not wearing any hats here, and this is just my personal
> opinion, but I  worry more about the virtual interim. We are
> clearly seeing more of them, and quality is essential there,
> as all presentations are remote. If they are
> valuable, they probably need to be supported and that would
> cost money. Would people pay a registration fee to attend a
> virtual interim ? Or, should we be looking for sponsors for
> that ?

Marshall,

I'm worried about both.  But there is a different way to look at
those virtual interims (or, indeed, non-virtual interims).  If
ADs continue to enforce the informal requirements that interims
are not used instead of regular IETF meetings, e.g., that WGs
who have decided to not meet at IETF don't get to hold interim
meetings, then presumably anyone attending the interim is paying
full freight.  Those who are attending the interim and F2F
meetings at IETF are paying registration fees for the latter.
Those who are attending the interim and the regular meetings
virtually/remotely are part of the remote participation
question.  A presenting remote participant causes all of the
costs to the IETF that a meeting attendee does except the cost
of cookies.  Interim meetings do generate extra costs in terms
of calendars and publication of minutes, etc., but I can't
imagine those costs are large for WGs that are meeting at
regular meetings anyway.

I do think that, as meeting attendance becomes more expensive
--not just in registration fees, but in travel and hotel costs,
time away from the office (a direct consequence of adding all
day Friday, decisions to hold meetings a considerable distance
from major international airports, etc.)-- it will be
increasingly important to avoid creating disincentives to
meeting attendance by making it both too easy and too cheap to
not attend.  Most of our argument for "people need to be there
all week" (include some pushback against day passes) involves
exchanges outside the WG meetings that people are primarily
present to attend.  A remote presenter who is present only for a
particular WG loses out on all of that -- a loss to those who do
it and to the rest of us.

I also think there is another possible bad consequence of
dedicating another WG participant or two to running WebEx (or
equivalent), or even monitoring Jabber for questions, during WG
sessions.  All things being equal and to the extent possible, I
think we would rather have active WG participants paying
attention to what is going on in the WG, not being distracted by
monitoring other input streams or paying attention to
technology.  It would be better if folks who were basically not
useful to the WG were performing those tasks, but those folks
are usually not the ones who volunteer.

Given both costs and consequences, perhaps it is reasonable to
permit, and even encourage, remote lurking in the interest of
openness.  But those who want to present, participate actively
in discussions, etc., should probably pay registration fees and
perhaps at least part of the marginal costs of supporting remote
participation, where "part" might have an upper bound of around
50% of the IAOC's estimate of overall meeting attendance costs
(at least average travel plus hotel).    If we created that
revenue stream, I could also imagine trying to hire local
students (for a small payment and/or full waiver of registration
fees) to take a WebEX training course, handle remote
participation, and, as a side-effect, get exposed to the IETF.
That would reduce burdens on active WG participants as well as
helping to offset general IETF costs.

> If costs go up, and nothing else is cut, the money has to come
> from one of these sources, or a new one has to be created...

Yes.  And taxing those who pay to attend, pay for the hotel (and
hence those commissions) and who, in addition, pay for travel
and contribute their time for the week in order to support those
who choose to not show up in person is not only inequitable, but
borders on folly (note that I said "choose", not "cannot attend"
-- if the IAOC wants to set up a waiver system for some people
under some conditions, I think that would be fine).

best,
    john

---

From: Eric Burger <[email-masked]>
Date: Mon, 19 Apr 2010 14:20:26 -0400

I have been sitting on the sidelines of this discussion for two reasons.  The first is to simply see what the community wants.  The second is that I did not look for like 30 hours and there was at that time 40+ messages, so I put off reading the list due to laziness and fear.  Unfortunately, I missed the First of the month to send these thoughts.  [This will become important later.]

Speaking purely as a WG chair, neither ISOC nor IAOC, I wonder what we are really trying to say here.  There seems to be a lot of sentiment that remote participation borders between an unfortunate situation, if the real expert cannot travel, to pure evil, if the individual "does not want" to participate in non-meeting activities like hallway conversations and Bar BOFs.

A very reasonable model, and one that has been incredibly successful, is to say that one can operate on the basis of in-person meetings only.  To go to the meeting, one must register and pay or be invited by the chair.  To make things easier for corporate-sponsored individuals, we can allow corporations to pay directly instead of hiding participation fees in the registration fee.  To simplify travel planning, we can have meetings at either the same location every time or rotate between three fixed locations.  Since we would be at the same locations, might as well save on meeting space by having our own offices in those locations.

Since we are perennially short on funds, we can co-host with an organization already steeped in that model. It does mean we will need to meet perpetually in Geneva, but there are worse places to go.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 20 Apr 2010 16:41:03 -0400

--On Wednesday, April 14, 2010 10:04:52 AM -0700 Paul Hoffman 
<[email-masked]> wrote:

> *Some of the problems were from WebEx participants not muting their mics
> when someone else was talking, as we discovered in the IPsecME meeting in
> Anaheim. There were maybe five people logged in on WebEx, and at one
> point the person speaking became almost inaudible. After I barked "If you
> are not speaking, mute your mic *now*", a few clicks were heard and the
> audio got noticeably better immedately.

If you are the webex session "host", you can mute for them.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 20 Apr 2010 16:55:57 -0400

> sort of density and with that kind of overlap.

Yeah, I noticed that too, as well as several of the other problems you 
noted.  And in at least one instance, I had to replace batteries in a 
wireless mic which died in the middle of a session.

Overall I was impressed with the effort put forth by our volunteers to make 
the audio and webex work.  With PSAV, not so much.  I was able to get 
acceptable results, with a bit of tweaking and periodic monitoring to make 
sure things were working well.  One thing I did as chair was to keep a 
phone dialed in to webex (but muted) so I could check periodically to 
verify everything was working correctly.

-- Jeff