Subject: Remote Participation at IETF's

From: David Frascone <[email-masked]>
Date: Wed, 25 Feb 2009 09:54:03 -0500

I could not attend the last IETF in person due to budget constraints at my
company.  But, Hannes managed to duct-tape together a barely working
teleconference using Skype.  I, again, am going to be unable to attend
(Thank you, recession), and I'd like to propose some better solutions for
the next few meetings, since it seems the recession is going to last a
while.

First, let me try to describe what Hannes did.  He put his laptop with Skype
very close to the microphone.  I could barely hear, but my voice was able to
be heard well.  So, using Skype in combination with the audio multicast was
*barely* usable (the multicast was delayed about 5 seconds from realtime)

The quality of the multicast audio is good -- but the delay makes
conversations impossible.  Also, the multicast picks up all microphones in
the room, whereas Hannes' solution did not pick up all microphones (or, the
quality was just so poor that I could only hear the presenters' voices
clearly)

So, I propose that we look for a supported teleconferencing solution for the
next few (and possibly ongoing) ietf meetings.

I have been on a few Webex calls, and I think their technology is very good.
 I would love to see the presentations at the same time as hear the
presenters.  And, with the large pipe to the 'net that is always present at
IETF meetings, I think the only hurdle is the actual cost of the Webex
service.  I think approaching Cisco (who bought Webex) about donating Webex
licenses would be a good start here.  If we could integrate Webex into
our existing streaming / audio system, remote participants could get a lot
more out of the meetings.

If the above is not possible, then I think we should look into better
integrating "private streaming" options into the meeting rooms.  Maybe
having line in/out options at the tables, so that we can use Skype, or other
real-time two way streaming protocols.

Or, lastly, we could drop back to the method that we used last time.  But,
with all the good technology out there -- there has to be a better way
to accommodate remote participation.

Comments?

-Dave

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 25 Feb 2009 10:16:16 -0800

I'll chime in here since I've actaully been working on this for 26 or
perhaps more meetings.

The recording/streaming component of the effort has been handed off to
the ietf's network contractor for the first time, so we'll see how that
goes, I've been working on this transition for sometime.

David Frascone wrote:
> 
> I could not attend the last IETF in person due to budget constraints
> at my company.  But, Hannes managed to duct-tape together a barely
> working teleconference using Skype.  I, again, am going to be unable
> to attend (Thank you, recession), and I'd like to propose some better
> solutions for the next few meetings, since it seems the recession is
> going to last a while.
> 
> First, let me try to describe what Hannes did.  He put his laptop
> with Skype very close to the microphone.  I could barely hear, but my
> voice was able to be heard well.  So, using Skype in combination with
> the audio multicast was *barely* usable (the multicast was delayed
> about 5 seconds from realtime)

So, first a definitional issue... The audio is not multicast, it is http
streamed over tcp. If it were multicast, experience says you likely
would not be recieving it due to the sad state of interdomain multicast
but playout would in fact be much faster given encoding overhead +
distance vector + decoding overhead was much shorter when we did things
that way and assuming the actually recieving it hardle was cleared... In
the current case there are buffers on basically every computer which
handles the audio stream including likely the largest one which is in
your streaming client itself... they all add up.

> conversations impossible.  Also, the multicast picks up all
> microphones in the room, whereas Hannes' solution did not pick up all
> microphones (or, the quality was just so poor that I could only hear
> the presenters' voices clearly)

The audio source is connected to the mixing bus in the room, since we
are attempting to stream the meeting, that is obviously necessary given
the sized of mosot of the rooms and the number of participants.

> So, I propose that we look for a supported teleconferencing solution
> for the next few (and possibly ongoing) ietf meetings.

Various individuals have been experimenting with both the technological
components necessary and the workflow associated with allowing remote
participation.

> I have been on a few Webex calls, and I think their technology is

I'd characterize webex as likely best of breed. That said using it in
this context is not free of additional work, and in fact the act of
mixing it into the in-room audio and vice-a-versa is non-trivial
(although in practical terms it requires the same equipment as any
approach attempting to achive the same goal).

> presenters.

Which requires the presenters all be part of the same webex session, on
each of their systems, or that they all use the same presentation
systems. heterogeneity has been an issue in the past, likely this is
something that could be solved by altering either requirements for a
meeting or workflow, but there's non-zero overhead, and if you were
holding out for informality in meeting or presentation style this will
certainly confer it a level of rigidity (and produce more work that
presumably gets done by wg chairs)

> And, with the large pipe to the 'net that is always present at IETF
> meetings, I think the only hurdle is the actual cost of the Webex
> service.

There are actually ongoing discussions with cisco on the subject, the
services is in fact quite expensive at scale, bandwdith at the meeting
site is not a serious issue, modula local participants who avail
themselves of the remote streams.

I would not as a general rule assume that bandwidth at an ietf meeting
is available to burn, in some venues it may be, in others it's merely
adequate, you do not pay for more than you have to. Obviously a meeting
like philly or paris where the host was able to draw on their own
provisioning capacity is differnt from a meeting not hosted by a
facilities provider.

> I think approaching Cisco (who bought Webex) about donating Webex
> licenses would be a good start here.  If we could integrate Webex
> into our existing streaming / audio system, remote participants could
> get a lot more out of the meetings.

There is the issue of floor control when mixing remote participants in
with local, the question of who handles that, and what level is necessary.

Remote presenters are actually fairly straight forward, the remote queue
at the mike is not, the sort of banter from the audience with a
presenter working on a topic that is possible in a small meeting is
quite unwelcome when it comes booming in from the ceiling without warning.

>  integrating "private streaming" options into the meeting rooms.
> Maybe having line in/out options at the tables, so that we can use
> Skype, or other real-time two way streaming protocols.

Having end users who don't know what they're doing insert themselves
into the room mix is extremely dangerous from a production
stand-point... In particular given you're dealing with full duplex
devices, it's trivial to produce your choice of feedback loops. The
mixers that we normall contract from the hotel are as a general rule
(other than the plenary) not sophisticated enough to be safely used to
insert and remove remote audio. further, because remote partipants are
varring distances from the meeting, have varying sets of equipment and
software and are being inserted into the in-room audio you have a
distributed echo-cancellation problem to deal with that is similar to
very large multi-pbx bridged conference calls and that requires both an
active signal processing component and some careful configuration.

	    output to remote > remote > input from remote
                    /			      /
                   /	/--------------------/
	      	adaptive noise gate
                    ^   |
                    |   V
in room source > multibus mixer -> room audio

> Or, lastly, we could drop back to the method that we used last time. 
> But, with all the good technology out there -- there has to be a
> better way to accommodate remote participation.

So, there's another non-technical issue. which is simply that if we
expend funds on making this work really well and as a result more people
choose to participate in a remote fashion, that actually erodes the
funding base that supports remote participation in the first place. Now
it is possible that that we can charge remote participants for the
privilege but given the (relatively) small number and their usage
patterns I think it's likely that that will reduce rather than enhance
usability and accessibility.

That's just my opinion of course, but I can pose the question to you:

Would you and about 50 other people for a given meeting be willing to
pay the equivalent of the regular meeting fee in order to attend
remotely the ietf meetings in order to have a much higher level of
service?

How about 150 people willing to pay $200?

Would you be willing to do that when that meetings were held in a
timezone that was ofset from your current one by 9-10 hours, or are
there really only one or two meetings that you're interested in having
that level service for.

I've spent some time thinking about these issues, I've experimented with
others in  order to solve them I'm willing to help experiment further in
the future I not longer really have time or funding to support the
streaming in a volunteer capacity at the level that we did for close to
a decade...

You can see the internet draft now long since expired that we used as a
jumping off point for the service that we have today here:

http://videolab.uoregon.edu/draft-jaeggli-ietftv-ng-01.txt

The more recent draft covering it's current status and some thoughts on
experimentation are here and I would appreciate some feedback on them...

http://www.ietf.org/internet-drafts/draft-jaeggli-ietf-streaming-media-status-00.txt

> Comments?

Just a few...

> -Dave

joel