Subject: [clue] My notes from Today's CLUE meeting (for the period when I scribed)

From: Marshall Eubanks <[email-masked]>
Date: Thu, 28 Jul 2011 19:34:58 -0400

CLUE - 1028 AM July 28 2011
Marshall Eubanks (second scribe)
took over from Magnus

------

Allyn Romanow - Clue framework

What we are doing here - telepresence deals with multiple streams, while our
standards deal with single streams.

Challenges - we want something

- immediately usable (or at least relatively quickly)
- extensible
- and simple and practical to implement

The Framework clusters around 2 concepts

- media capture information that needs to be passed
- how the provider figures out what streams to send

process - provider provides capabilities
        - consumer choses from these

optimization - before negotiation, the consumer may send info about itself
to the
provider, so the provider can tailor what it provides.

Properties

- Media capture
- encode groups
- simultaneous transmission sets

I want to take a minute to set context - any proposed framework was going to
be difficult to communicate. We thought we should do so in stages and start
simple. We would like it if people would focus on the concepts first.

------
Mark Duckworth

Media Capture and Attributes

A media capture is a source of audio or video media.

They can be

- media from a camera or a microphone (a capture device)

- media from a combination of media devices

- or, this could be done remotely.

A capture set is a way to group media captures that have some relations

Some attributed include things like

- is the video auto switched or composed ?
- is the audio mixed ?
- audio channel format (mono/stereo…)
- what is the spatial scale / image width on video

Attributes include a "purpose" - say, main versus presentation

I want to introduce a capture scene -

imagine a given scene with people - cameras - camera views

types include

- one camera per screen
- merging cameras in some fashion
- switched based on voice with a composed PiP

etc.

Chris - you don't assume that the whole scene is always shown

Mark - of course

Roni - What about other models ? (lists some)

Mark - this is just one example.

A capture set is a representation of a group of video capture. It has N
"rows." Each row is a capture set. Ordering within the rows important - it's
how the left to right order is imposed.

[big discussion of capture sets and what they mean and how they might work
dynamically, which alas I took part so didn't capture]

Cullen - when we charted this WG

Allyn - we were trying to start with something simple

Stephan Botcho - the goal here is to achieve interoperability. Having some
approximate idea of adjacency may be more powerful

Roni - this talks about a simple architecture

Christian ? - we still have a concept of left right ?

Stephan Wenger - I am willing to spend work, but I am not willing to let you
off the hook when there are requirements that are relevant for me.

Mark - Matching audio and video - when they are part of the same capture set
- that includes time synchronization and spatial relationships

Spatial relationships - audio direction should roughly match video
directions

In the audio, we are calling this audio channel formats - a receiver can map
these
into its loudspeakers to approximate the spatial relationship, in a way
better than just going to mono, but not requiring identical audio formats.

Allyn - the point is whether or not the draft deals with everything we need
to capture the framework, not whether it deals with all of the details.

------
Andy Pepperell

Choosing streams

Basic Message flow

media stream consumer and media stream provider

(of course, typically side each has both)

msc communicates with msp

msc : consumer capability advertisement
msp : media capture advertisement

Initial message msc : consumer capability advertisement
(AKA "the hint")

Physical factors
User preferences
Software limitations
etc.

Next (the second message,from msp to msc) is the media capture advertisement
from the msp

- most recently received consumer capability advertisement
- provider fixed parameters, such as the number of cameras
- dynamic factors - active speaker, presentation source status,
- simultaneous transmission sets, etc.

Third message (msc to msp)

Stream configure message from the msc

based on media capture advertisement
consumer fixed characteristics
dynamic factors

this is the trigger for actual media transfer from the provider

question - why not us the terms sender and receiver ?

Andy - we thought this was a little different case and that might confuse
people.

Mark - and, this is not the sending and receiving of media

Andy - simultaneous transmission sets

suppose that the same camera is a digital zoom of one sub-scene, and also
provides the entire scene - that's why you need simultaneous transmission
sets

Encoding groups  - part of the media capture set advertisement
by the media stream provider. Each capture has an associated

Encoding group structure - within an encoding group, there is a possibility
of multiple encode or multiple potential encodes

the usual sort of video encode attributed (advertised by the provider to the
consumer)
(these are the usual sorts of stuff, bandwidth, max bandwidth, etc.)

Roni - from the consumer side you are talking about screens.

You also have to some way of linking encoding group with a specific codec.

Marshall Eubanks : So, if something changes in the middle of a session, to
change things you will have to have the provider send a new media capture
advertisement to the consumer, which will have to then send a new Stream
configure message, to get the changed stream.

Andy : Yes

Marshall : So the consumer will have to be listening to the provider for
MCAs at any time?

Andy : Yes.

Marshall : And, of course, you will need error messages.

Andy : Of course. The msc might get it wrong.

Meeting ended at 11:30 AM EDT.