Subject: [clue] Capture scene clarifications

From: Christian Groves <[email-masked]>
Date: Fri, 23 Nov 2012 14:55:55 +1100

Hello,

Paul K reminded me that I have an action point from the interim meeting 
regarding my draft <draft-groves-clue-scene-clarifications>. At the 
meeting it seems that there was general agreement with the intent of the 
proposals:
    a)      A scene represents an area where the capture devices are
            spatially related, i.e. a presentation that shares no spatial
            relation with a video is a separate scene.

    b)      A capture scene entry thus represents alternate
            representations of a complete scene.  The provider is the one
            that determines what a complete scene is.  A consumer choses
            a capture scene entry from the scene in the knowledge that it
            represents the entire scene.

    d)      A consumer shall chose a capture scene entry rather than
            choosing individual captures from multiple entries.  This
            does not mean that an consuming endpoint must render all the
            captures.  What is locally rendered and how is a local
            decision.

I think the framework draft could be enhanced in a few places to make 
the intent of CLUE clearer. Below are some suggested updates.

1) Introduce a new definition for "Scene". The capture scene definition 
and several places mention "scene" but there's no explanation. I propose 
the following definition to be added to section 3 of the draft:
Scene: Represents an area where the capture devices are spatially 
related. Non-spatially related devices exist in different scenes.

2) Update the definition of "Capture scene entry" to make it clearer 
that it represents an entire scene. Proposed text is:
*Capture Scene Entry: a list of media captures of the same media type
    that together form one representation of an entire capture scene.

3) Update section 6.2 on Capture scenes to reflect the above intents. 
Changes proposals are:
- Section 6.2 2nd paragraph:
"A capture scene is a structure representing the <<entire>> scene that is
    captured by a collection of<<spatially related>>capture devices.  A 
capture scene..."

- Section 6.2 3rd paragraph:
  "A provider may advertise multiple capture scenes or just a single
    capture scene.<<What constitutes an entire scene is up to the 
provider.>> A media provider might typically use one capture
    scene for main participant media and another capture scene for a
    computer generated presentation...."

- Section 6.2 4th paragraph:
"A media provider arranges media captures in a capture scene to help
    the media consumer choose which captures it wants.  The capture scene
    entries in a capture scene are different alternatives the provider is
    suggesting for representing the entire capture scene.<delete rest of 
paragraph>".

- Section 6.2 5th paragraph:
  "Media captures within the same capture scene entry must be of the
    same media type - it is not possible to mix audio and video captures
    in the same capture scene entry, for instance.  The provider must be
    capable of encoding and sending all media captures in a single entry
    simultaneously."  <delete rest of paragraph>

- section 6.2 New paragraph dealing with consumer behaviour after the 
7th paragraph.
  "A consumer may receive an advertisement with multiple capture scenes. 
A consumer may choose to receive any number of capture scenes. An 
advertised capture scene it may contain one of more capture scene 
entries that may contain one of more media captures. A consumer may 
choose an capture scene entry in the knowledge that it is a complete 
representation of the scene for a particular media type and that all 
media captures are spatially related. For a particular media type the 
consumer shall choose one capture scene entry rather than choosing 
individual captures from multiple capture scene entries. However this 
does not mean that a consuming endpoint must render all the media 
captures. What is locally rendered and how is a local decision."

4) The framework uses the terms "media provider" and "provider". We 
probably should be consistent in the use of the term throughout the 
framework.

5) Related to the "capture group" concept that didn't seem to get 
support is the ordering in the Capture scene, Capture scene entry, Media 
captures. There was some discussion that people had assumed some sort of 
prioritisation / ordering related to these structures. i.e. If I have a 
capture scene entry VC0, VC1, VC2 do I assume they should be rendered 
left to right or is there nothing to be assumed? I take it is the later 
understanding put I didn't see anything in the framework regarding what 
should be assumed. We probably should also add something for that.

Thoughts?

Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Sat, 24 Nov 2012 14:53:58 +0000

Hi Christian,

If one intention is to clarify that a capture scene entry represents a while scene, why not remove "capture", and simply call it "scene entry", or something...

And, as the defintion for "scene" already talks about media captures, we could remove "media capture" from the definition of "scene entry", and say:

"A list of alternative representations of a single scene"

...or something like that.

I think things would be more clear if we try to keep "capture" and "scene" apart from each other.

Regards,

Christer 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christian Groves
Sent: 23. marraskuuta 2012 5:56
To: [email-masked]
Subject: [clue] Capture scene clarifications

Hello,

Paul K reminded me that I have an action point from the interim meeting regarding my draft <draft-groves-clue-scene-clarifications>. At the meeting it seems that there was general agreement with the intent of the
proposals:
    a)      A scene represents an area where the capture devices are
            spatially related, i.e. a presentation that shares no spatial
            relation with a video is a separate scene.

I think the framework draft could be enhanced in a few places to make the intent of CLUE clearer. Below are some suggested updates.

1) Introduce a new definition for "Scene". The capture scene definition and several places mention "scene" but there's no explanation. I propose the following definition to be added to section 3 of the draft:
Scene: Represents an area where the capture devices are spatially related. Non-spatially related devices exist in different scenes.

2) Update the definition of "Capture scene entry" to make it clearer that it represents an entire scene. Proposed text is:
*Capture Scene Entry: a list of media captures of the same media type
    that together form one representation of an entire capture scene.

3) Update section 6.2 on Capture scenes to reflect the above intents. 
Changes proposals are:
- Section 6.2 2nd paragraph:
"A capture scene is a structure representing the <<entire>> scene that is
    captured by a collection of<<spatially related>>capture devices.  A capture scene..."

    capture scene.<<What constitutes an entire scene is up to the provider.>> A media provider might typically use one capture

- section 6.2 New paragraph dealing with consumer behaviour after the 7th paragraph.
  "A consumer may receive an advertisement with multiple capture scenes. 
A consumer may choose to receive any number of capture scenes. An advertised capture scene it may contain one of more capture scene entries that may contain one of more media captures. A consumer may choose an capture scene entry in the knowledge that it is a complete representation of the scene for a particular media type and that all media captures are spatially related. For a particular media type the consumer shall choose one capture scene entry rather than choosing individual captures from multiple capture scene entries. However this does not mean that a consuming endpoint must render all the media captures. What is locally rendered and how is a local decision."

4) The framework uses the terms "media provider" and "provider". We probably should be consistent in the use of the term throughout the framework.

5) Related to the "capture group" concept that didn't seem to get support is the ordering in the Capture scene, Capture scene entry, Media captures. There was some discussion that people had assumed some sort of prioritisation / ordering related to these structures. i.e. If I have a capture scene entry VC0, VC1, VC2 do I assume they should be rendered left to right or is there nothing to be assumed? I take it is the later understanding put I didn't see anything in the framework regarding what should be assumed. We probably should also add something for that.

Thoughts?

Regards, Christian
_______________________________________________
clue mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/clue

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 26 Nov 2012 10:14:59 -0500

Christian,

Thanks for providing these. They all seem reasonable to me, with one 
exception:

On 11/22/12 10:55 PM, Christian Groves wrote:

[snip]

> media captures are spatially related.

Above seems ok.

> For a particular media type the

This doesn't cover the intent as I understand it.

We certainly don't intend to require that a receiver request captures 
that it intends to ignore. Selecting a subset of the captures in a 
capture scene entry is ok if that is what the receiver wants to do, even 
though it may result in an incomplete rendering of the scene.

Also, it is ok to select some or all of the captures from multiple 
capture scene entries if that is what you really want to do. This might 
be unusual behavior for an endpoint, but not forbidden. It might be 
entirely reasonable for a MCU to do this.

This is all in the spirit of the sender not trying to second guess what 
the receiver might want to do.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 26 Nov 2012 19:29:58 +0000

Hi, 

> We certainly don't intend to require that a receiver request captures that it intends to ignore. Selecting a subset 
> of the captures in a capture scene entry is ok if that is what the receiver wants to do, even though it may result 
> in an incomplete rendering of the scene.

I guess I would be ok with that.

> Also, it is ok to select some or all of the captures from multiple capture scene entries if that is what you really 
> want to do. This might be unusual behavior for an endpoint, but not forbidden. It might be entirely reasonable for a 
> MCU to do this.

Doesn't that complicate things, for no obvious reason?

How does one know that the provider even is able to submit captures from different entries (e.g. maybe the same physical camera is used for different things, depending on which capture scene entry is "active")? We would need to have lots of ANDs, ORs and XORs...

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 26 Nov 2012 14:57:38 -0500

> Doesn't that complicate things, for no obvious reason?

Some reasons:

- an MCU that selects multiple scene entries from an endpoint may be 
able to advertise more flexible alternatives in its own advertisements.

- an endpoint might find that every capture scene entry has more 
captures than it can handle, so it is forced to choose some incomple 
representation of the scene. It may be that choosing selected captures 
from different scene entries might be more useful to it than selecting a 
subset of captures from a single entry.

- a user of an endpoint with good flexible UI controls may be able to 
browse the entire advertisement and pick what he likes.

> How does one know that the provider even is able to submit captures from different entries (e.g. maybe the same physical camera is used for different things, depending on which capture scene entry…

The simultaneous transmission set is supposed to provide this information.

I thought it was good when Christian came along and gave a fresh 
perspective on this without having lived through all the discussions 
that led up to the framework as it is. So he was pointing out holes 
ambiguities in the text.

But I find it very interesting that even people who have been involved 
for a long time are disagreeing about what is supposedly settled ground.

	Thanks,
	Paul

---

From: Christian Groves <[email-masked]>
Date: Tue, 27 Nov 2012 15:12:41 +1100

Hello Christer,

I'm all for simplifying things. If we can use one term throughout the 
draft instead of two closely related ones I think it would be good. 
However can we completely decouple "scene" and "capture" after all a set 
of captures basically constitutes a scene?

Regards, Christian

---

From: Christian Groves <[email-masked]>
Date: Tue, 27 Nov 2012 15:50:48 +1100

Hello Paul,

I had the same thoughts as Christer. Picking a subset is something that 
I could live with, cherry picking captures from different capture scene 
entries I think is complicating the issue. The framework says that 
provider must be able to supply  media related to all the captures 
within a capture scene entry. If the consumer chooses different captures 
then the provider may not be able to supply these? Do we then need an 
extra message for the provider to signal to the consumer that it can't 
honour its choice?

Regards, Christian

On 27/11/2012 6:57 AM, Paul Kyzivat wrote:
>>> We certainly don't intend to require that a receiver request 
>>> captures that it intends to ignore. Selecting a subset
>>> of the captures in a capture scene entry is ok if that is what the 
>>> receiver wants to do, even though it may result
>>> capture scene entries if that is what you really
>>> want to do. This might be unusual behavior for an endpoint, but not 
>>> forbidden. It might be entirely reasonable for a
>>> MCU to do this.
>>
[CNG] Are you inferring that CLUE is an end-to-end protocol? I would see 
in this case the MCU would simply construct its own advertisement. The 
original provider can only provide what it can provide irrespective of 
what a MCU would subsequently offer.
> from different scene entries might be more useful to it than selecting 
> a subset of captures from a single entry.
[CNG] How will the consumer know that it can provide these captures from 
different entries at once? Given the current attributes for captures I'm 
not sure that a consumer would know enough to determine that a group of 
different captures from multiple capture scene entries constitutes an 
entire scene. If we say the consumer is smart enough to determine this 
then why bother with the capture scene entry concept?

> browse the entire advertisement and pick what he likes.
[CNG] In theory yes but at this point there won't be any video/audio so 
unless the user is knowledgeable in XML I don't see that they'd go down 
to the individual capture level.
>
>> How does one know that the provider even is able to submit captures 
>> from different entries (e.g. maybe the same physical camera is used 
>> for different things, depending on which capture scene entry is 
>> "active")? We would need to have lots of ANDs, ORs and XORs...
>
[CNG] This does say that the consumer "must make sure that it chooses one
    and not more of the mutually exclusive sets"? Also is the sending of 
a simultaneous transmission set mandatory?

> that led up to the framework as it is. So he was pointing out holes 
[CNG] Well I have followed discussions on the mailing list, I think it 
was attending the Andover CLUE interim where I think it became apparent 
people had slightly different understandings.
> for a long time are disagreeing about what is supposedly settled ground.
[CNG] :-)

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 27 Nov 2012 00:24:57 -0500

> Hello Paul,

Disclaimer - AFAIK I am simply restating the intent of the framework as 
it stands. These were not my concepts. But I think I have assimilated 
the intent.

So it is certainly possible to recast this stuff, but then I think there 
will need to be some further discussion to fully understand the 
ramifications of such a change.

> entries I think is complicating the issue.

Certainly one can argue about how one would know enough to make such a 
choice. But I don't see why it is a "complication" from the point of 
view of the provider.

> The framework says that

The simultaneous transmission sets are intended to indicate what can be 
transmitted simultaneously. The advertiser should thus be able to 
transmit any selection that is in accord with the advertisement.

The specification that the provider must be able to supply media related 
to all captures within a scene entry is simply a further constraint on 
the provider.

Once the provider has constructed its advertisement to reflect its 
constraints, it should be no extra complication to supply any conforming 
selection, regardless of which scene entries the captures come from.

Perhaps making a limitation of only selecting from one scene entry per 
scene might simplify the advertisement - perhaps eliminating the need 
for the simultaneous transmission set. Or it might simplify the choice 
of configuration. But neither of these seems like enough of a 
complication to worry about.

So ISTM that such a change will need to be based on esthetics and 
conceptual simplicity.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 27 Nov 2012 08:48:38 +0000

Hi,

>>>> individual captures from multiple capture scene entries. However 
>>>> this does not mean that a consuming endpoint must render all the 
>>>> media captures. What is locally rendered and how is a local decision."
>>>
>>> capture scene entry is ok if that is what the receiver wants to do, even though it may result in an incomplete rendering of the scene.
>>
>>> capture scene entries if that is what you really want to do. This 
>>> might be unusual behavior for an endpoint, but not forbidden. It might be entirely reasonable for a MCU to do this.
>>
> - an MCU that selects multiple scene entries from an endpoint may be able to advertise more flexible alternatives in its own advertisements.
>
> - an endpoint might find that every capture scene entry has more captures than it can handle, so it is forced to choose some incomple representation of the scene. It may be that choosing selected captures from different scene entries might be more 
>useful to it than selecting a subset of captures from a single entry.
>
>- a user of an endpoint with good flexible UI controls may be able to browse the entire advertisement and pick what he likes.

Those reasons are all theoretical, in my view. If the provider advertises a wide range of capture scene entries, I am sure the consumer will find a suitable one - especially if it also can selects specific captures of the capture scene entry.

>> How does one know that the provider even is able to submit captures from different entries (e.g. maybe the same physical camera is used 
>> for different things, depending on which capture scene entry is "active")? We would need to have lots of ANDs, ORs and XORs...
>
> I thought it was good when Christian came along and gave a fresh perspective on this without having lived through all the discussions that led up to the framework as it is. So he was pointing out holes ambiguities in the text.
>
> But I find it very interesting that even people who have been involved for a long time are disagreeing about what is supposedly settled ground.

I think I DID express concern about selecting stuff from different capture scene entries at that last interim.

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 27 Nov 2012 09:06:15 +0000

Hi,

> Disclaimer - AFAIK I am simply restating the intent of the framework as it stands. These were not my concepts. But I think I have assimilated the intent.
>
> So it is certainly possible to recast this stuff, but then I think there will need to be some further discussion to fully understand the ramifications of such a change.
>
>> I had the same thoughts as Christer. Picking a subset is something 
>> that I could live with, cherry picking captures from different capture 
>> scene entries I think is complicating the issue.
>
> Certainly one can argue about how one would know enough to make such a choice. But I don't see why it is a "complication" from the point of view of the provider.

It makes the procedures more complex.

Without the option to "cherry picking", the provider only needs to say:

"These are my CSEs (Capture Scene Entries). Please pick one, and then till me which captures you want."

With the option to cherry pick, the provider would in addition also need to be able to say:

"BTW, if you choose CSE_X, and capture CAP_X, then from CSE_Y can only choose captures CAP_Y1, CAP_Y2, CAP_Y3 - unless you also from CSE_W chooses capture CAP_W1, in which case you can only choose CAP_Y2 and CAP_Y3 from CSE_Y".

Etc etc etc etc etc.

I could become very complicated, depending on how far we want to go.

>> within a capture scene entry. If the consumer chooses different 
>> captures then the provider may not be able to supply these? Do we then 
>> need an extra message for the provider to signal to the consumer that 
>> it can't honour its choice?
>
> The simultaneous transmission sets are intended to indicate what can be transmitted simultaneously. The advertiser should thus be able to transmit any selection that is in accord with the advertisement.
>
> The specification that the provider must be able to supply media related to all captures within a scene entry is simply a further constraint on the provider.
>
> Once the provider has constructed its advertisement to reflect its constraints, it should be no extra complication to supply any conforming selection, regardless of which scene entries the captures come from.
>
> Perhaps making a limitation of only selecting from one scene entry per scene might simplify the advertisement - perhaps eliminating the need for the simultaneous transmission set. Or it might simplify the choice of configuration. But neither of these 
> seems like enough of a complication to worry about.

I disagree. I think it can become very complicated.

Simultaneous transmission sets is perhaps something we could add in future, if there is a need.

Also, if people are afraid that things will become too easy, please join the discussions about SDP and multiple streams per m- line :)

Regards,

Christer

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 27 Nov 2012 09:08:23 +0000

Hi,

>I'm all for simplifying things. If we can use one term throughout the draft instead of two closely related ones I think it would be good. 
>However can we completely decouple "scene" and "capture" after all a set of captures basically constitutes a scene?

Yes, and in the definition we say that a scene is a set of captures. Then we don't need to talk about "capture scene entry", which at least to me sounds confusing :)

Regards,

Christer

On 25/11/2012 1:53 AM, Christer Holmberg wrote:
> Paul K reminded me that I have an action point from the interim 
> meeting regarding my draft <draft-groves-clue-scene-clarifications>. 
> At the meeting it seems that there was general agreement with the 
> intent of the
> proposals:

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 27 Nov 2012 11:39:41 +0000

Hi,

A little more on this:

Instead of having simultaneous transmission sets, I think it would be more important to have something like "simultaneous capture scene sets".

Because, AFAIK, the video captures and a presentation capture might be part of separate capture scene entries (as the presentation does not have any spatial relationship with the video).

But, this would not be cherry picking individual captures from different CSEs, but rather selecting different CSEs - and being able to advertise that one can provide CSEs simultaneously.

Regards,

Christer

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Christer Holmberg
Sent: 27. marraskuuta 2012 11:06
To: Paul Kyzivat; [email-masked]
Subject: Re: [clue] Capture scene clarifications

Hi,

>> that I could live with, cherry picking captures from different 
>> capture scene entries I think is complicating the issue.
>

Etc etc etc etc etc.

>> captures then the provider may not be able to supply these? Do we 
>> then need an extra message for the provider to signal to the consumer 
>> that it can't honour its choice?
>
> scene might simplify the advertisement - perhaps eliminating the need for the simultaneous transmission set. Or it might simplify the choice of configuration. But neither of these seems like enough of a complication to worry about.

Regards,

Christer

---

From: Christian Groves <[email-masked]>
Date: Wed, 28 Nov 2012 15:16:07 +1100

Hello Christer,

A simultaneous set applies across all capture scenes in an advertisement 
so would cover the presentation / video case. I assume each simultaneous 
set can have a mix of media types, although this is not explicit in the 
framework?

I agree however if a consumer isn't allowed to select individual 
captures from multiple capture scene entries and scenes then we could 
probably simplify the use of simultaneous sets to the scene level.

Regards, Christian

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 28 Nov 2012 07:28:59 +0000

Hi,

> A simultaneous set applies across all capture scenes in an advertisement so would cover the presentation / video case. I assume each simultaneous set can have a mix of media types, although this is not explicit in the framework?

Sure. But, my understanding was that all captures within a scene have a spatial relationship. So, for that reason a presentation would have to be part of a separate scene (unless, of course, the presentation does have some kind of spatial relationship with the other captures).

> I agree however if a consumer isn't allowed to select individual captures from multiple capture scene entries and scenes then we could probably simplify the use of simultaneous sets to the scene level.

I would support such approach.

And, in practice this would still allow a consumer to select individual captures, by selecting multiple scenes, and then select captures from each scene. BUT, we would not need to describe which captures can be simultaneously provided, because if a provider indicates that SCENES can be provided simultaneously it automatically means that all captures within those scenes can be simultaneously provided.

Regards,

Christer

Regards, Christian

On 27/11/2012 10:39 PM, Christer Holmberg wrote:
>>> then need an extra message for the provider to signal to the 
>>> consumer that it can't honour its choice?
>> Perhaps making a limitation of only selecting from one scene entry 
>> per scene might simplify the advertisement - perhaps eliminating the need for the simultaneous transmission set. Or it might simplify the choice of configuration. But neither of these seems like enough of a complication to worry about.
> Also, if people are afraid that things will become too easy, please 
> join the discussions about SDP and multiple streams per m- line :)
>
> Regards,
>
> Christer
>

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 28 Nov 2012 13:04:29 -0500

Consider a case where an endpoint has a camera that can be used either 
to show part of the room or as a document camera. But it can't be used 
for both.

This might be a case where the presentation stream should be included in 
the same scene as the room. (Because it is a view of a whiteboard that 
has a physical location within the room.) But maybe not - it might still 
be desirable to include it in a separate scene. Lets assume that is the 
intent.

Then in this case there will be a mutual exclusion between captures in 
one or more scene entries in the room scene, and a capture in a scene 
entry in the presentation scene.

So there is still a need for mutual exclusion sets.

Or maybe a more radical change in structure. As I have tried to explain 
before, scene entries present alternative representations of a scene. 
But there is no way for the advertiser to indicate tradeoffs between 
captures in different scenes.

Perhaps we should consider allowing an "entry" to contain captures from 
multiple scenes, and a single list of such entries, rather than a list 
per scene. This would permit the advertiser to make its own decision of 
tradeoffs render its advertisement as a whole.

If we did that, then a scene would simply be an coordinate space and a 
set of captures that belong to that coordinate space.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 28 Nov 2012 19:52:41 +0000

Hi,

> be desirable to include it in a separate scene. Lets assume that is the

Well, then you have 3 different scene entries:

1. Room
2. Presentation
3. Room + presentation (where the presentation is the view of the whiteboard)

Then you indicate that you can provide the following scenes: 1 OR 2 OR 3 OR (1 AND 2).

>So there is still a need for mutual exclusion sets.

I think providing different scene alternatives is enough.

Sure, one can come up with examples which, if you need a separate scene to cover everything, would result in a large numer of scene entries. But, I don't really think it's needed in real-life.

>captures in different scenes.

Well, my opinion is that there is no need for that. It's enough to indicate which scenes (including all associated captures) can be provided simultaneously, as in the example above. 

Some telepresence vendor can correct me if I'm wrong, but from my personal experience that would cover existing use-cases, and probably many future ones too.

(We can make sure the protocol is extendable, if someone wants to add more stuff in future, but I don't think it's needed in order the first shipment.)

Regards,

Christer

        Thanks,
        Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 28 Nov 2012 15:50:08 -0500

Where is everybody else??? We need other people in this discussion.

More inline.

> Then you indicate that you can provide the following scenes: 1 OR 2 OR 3 OR (1 AND 2).

I don't think so. Above is mixing up scenes and scene entries.

Lets get a little more specific. Assume the room has three total cameras:
- a "left" camera
- a "right" camera
- a camera that can be moved & focused, to provide either
   a view of the whole room, or a view of a document platform.
   The whole room view may show people not easily and fully viwed
   in the left/right cameras, but doesn't show people life size.

So there are four potential captures:
- "left" capture
- "right" capture
- "whole room" capture
- "presentation" capture

There will be two scenes in the room, a "room" scene and a 
"presentation" scene. The "room" scene has multiple entries, while the 
"presentation" scene has only one entry.

"room" scene:
- entry: "left", "right", "whole room"
- entry: "left", "right"
- entry: "whole room"

"presentation" scene:
- entry: "presentation"

Mutual exclusion sets:
- "whole room", "presentation"

Note that the above gives no guidance to the receiver that can handle 
only a single capture. Should it take the presentation or the whole room?

> I think providing different scene alternatives is enough.

I think I still don't understand what you are proposing.

> (We can make sure the protocol is extendable, if someone wants to add more stuff in future, but I don't think it's needed in order the first shipment.)

For the example I gave above, using what I am proposing here, there 
would be a single set of entries covering both scenes:

"room" scene:
- entry: "left", "right", "whole room"
- entry: "left", "right", "presentation"

The above are the only alternatives that truely represent the entire 
room. They don't give any guidance about which to choose if you can't 
handle three captures. It would be possible to look at attributes of the 
captures and make some guesses.

Alternatively, it would be possible for the advertiser to make its own 
decisions about what tradeoffs to make when fewer captures can be 
handled. If so, we might see:

"room" scene:
- entry: "left", "right", "whole room"
- entry: "left", "right", "presentation"
- entry: "left", "right"
- entry: "whole room"
- entry: "presentation"

None of the two or one capture entries is really acceptable. It may be 
necessary for the client to switch among them.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 29 Nov 2012 09:46:13 +0000

Hi,

>Where is everybody else??? We need other people in this discussion.

Especially those who actually design telepresence rooms.

>>> Consider a case where an endpoint has a camera that can be used 
>>> either to show part of the room or as a document camera. But it can't 
>>> be used for both.
>>>
>>> This might be a case where the presentation stream should be included 
>>> in the same scene as the room. (Because it is a view of a whiteboard 
>>> that has a physical location within the room.) But maybe not - it 
>>> might still be desirable to include it in a separate scene. Lets 
>>> assume that is the intent.
>>

I guess I messed up terminology. Sorry for that.

I was meant to say that we have 3 different SCENES.

(Perhaps one could argue that 1) and 3) are capture scene entries for the same scene, but I don't think it is).

>So there are four potential captures:

Yes.

>There will be two scenes in the room, a "room" scene and a "presentation" scene. The "room" scene has multiple entries, while the "presentation" scene has only one entry.
>
>
>"room" scene:

Yes.

>- entry: "presentation"

Yes.

>- "whole room", "presentation"

Yes. 

>Note that the above gives no guidance to the receiver that can handle only a single capture. Should it take the presentation or the whole room?

It is of course good if we can provide a mechanism to convey information, but the actual decision is an application issue.

>>> Then in this case there will be a mutual exclusion between captures 
>>> in one or more scene entries in the room scene, and a capture in a 
>>> scene entry in the presentation scene.
>>>

I think I am proposing exactly what you described above :)

>>>> Or maybe a more radical change in structure. As I have tried to 
>>>> explain before, scene entries present alternative representations of a scene.
>>> (We can make sure the protocol is extendable, if someone wants to add 
>>> more stuff in future, but I don't think it's needed in order the first 
>>> shipment.)
>>
>> For the example I gave above, using what I am proposing here, there would be a single set of entries covering both scenes:
>>
>> "room" scene:
> The above are the only alternatives that truely represent the entire room. They don't give any guidance about which to choose if you can't handle 
> three captures. It would be possible to look at attributes of the captures and make some guesses.

Yes.

>Alternatively, it would be possible for the advertiser to make its own decisions about what tradeoffs to make when fewer captures can be handled. If so, we might see:
>
>"room" scene:
> None of the two or one capture entries is really acceptable. It may be necessary for the client to switch among them.

Sure. But, again, I don't we need to provide a mechanism for the receiver to create his own entry, by cherry picking captures from different entries. Because, that would mean that the provider, for each capture in each entry, need to indicate how/if a capture can be provided together with captures in other entries.

Regards,

Christer

---

From: Christian Groves <[email-masked]>
Date: Fri, 30 Nov 2012 10:58:51 +1100

Hello,

> (Perhaps one could argue that 1) and 3) are capture scene entries for the same scene, but I don't think it is).
[CNG] Isn't there still only two scenes? Is the room + presentation 
spatially related? If not then they are two separate scenes.
> It is of course good if we can provide a mechanism to convey information, but the actual decision is an application issue.
[CNG] I think this perhaps is a fundamental point. Its hard to define 
the protocol if we don't at least have a good understanding of what the 
sender and the receiver will do with the information. The idea behind 
the capture attribute draft was to give enough information to the 
receiver to decide what capture/s (and scenes) is more important to it. 
My assumption is that the receiver will parse a CLUE advertisement in 
the following manner:
1a. For each scene parse the capture scene entries
     1b. For each media type:
          1c. Check capture attributes to determine if best match
              according to local policy. i.e. best match according
              to content and spatial information.
2. The result of stage 1 will be to identify:
     a) a list of scenes that the receiving would like
     b) a for each scene a prioritised list of capture scene entries.
3. Based on stage 2 the receiver will then check the prioritised 
scenes/capture scene entries against any simultaneity requirements.
4. The resultant selected captures are then checked against encoding 
requirements.
5. The receiver then replies to the advertiser the scenes/capture scene 
entries that it wishes to receive.

Is this what others had in mind?

>> None of the two or one capture entries is really acceptable. It may be necessary for the client to switch among them.
[CNG] Are you proposing that the advertiser provides these alternatives 
or is this what the consumer constructs?

---

From: Christer Holmberg <[email-masked]>
Date: Fri, 30 Nov 2012 07:54:33 +0000

Hi,

>>>>> either to show part of the room or as a document camera. But it 
>>>>> can't be used for both.
>>>>>
>>>>> This might be a case where the presentation stream should be 
>>>>> included in the same scene as the room. (Because it is a view of a 
>>>>> whiteboard that has a physical location within the room.) But maybe 
>>>>> not - it might still be desirable to include it in a separate 
>>>>> scene. Lets assume that is the intent.
> [CNG] Isn't there still only two scenes? Is the room + presentation spatially related? If not then they are two separate scenes.

In the Room + Presentation alternative they ARE spatially related.

> [CNG] I think this perhaps is a fundamental point. Its hard to define the protocol if we don't at least have a good understanding of what the sender and the 
> receiver will do with the information. The idea behind the capture attribute draft was to give enough information to the receiver to decide what capture/s (and scenes) 
> is more important to it. 

The spatial information of course need to be there.

Then, as you say, we either have to provide content information about the capture, OR provide enough entries so that virtually any receiver will find one which fits its capabilities.

> My assumption is that the receiver will parse a CLUE advertisement in the following manner:
> 3. Based on stage 2 the receiver will then check the prioritised scenes/capture scene entries against any simultaneity requirements.
> 4. The resultant selected captures are then checked against encoding requirements.
> 5. The receiver then replies to the advertiser the scenes/capture scene entries that it wishes to receive.
>

Steps 1-4 are of course implementation dependent, but my assumption is also that step 5 is the outcome :)

>>>>>> But there is no way for the advertiser to indicate tradeoffs 
>>>>>> between captures in different scenes.
>>>>> (We can make sure the protocol is extendable, if someone wants to 
>>>>> add more stuff in future, but I don't think it's needed in order 
>>>>> the first
>>>>> shipment.)
>>> room. They don't give any guidance about which to choose if you can't handle three captures. It would be possible to look at attributes of the captures and make some guesses.
>> Yes.
>>
> [CNG] Are you proposing that the advertiser provides these alternatives or is this what the consumer constructs?

I guess Paul should reply, but my opinion is still that the receiver shall not construct entries. It shall choose from the advertised entries.

Otherwise, the provider will have to convey information about how entries can be constructed (how captures from different entries can be combined into a new entry, etc), and I see no need for that.

If an advertiser provides enough entries, I'm sure the receiver will find a suitable one :)

Regards,

Christer