Subject: [clue] draft-romanow-clue-telepresence-requirements-02: Spatial Capture Description - Reqm 1, 2 and 3

From: Erlendur Karlsson <[email-masked]>
Date: Wed, 11 May 2011 18:36:32 +0200

Hi,

I and some of my colleages have looked over the Spatial Capture Description Requirements (1,2 and 3) and we have some comments and suggestions on those.

These requirements mandate means of describing spatial arrangements of video images and audio signals, but there is no mention of what particular functionality they should enable or support. Due to this any spatial arrangement description will automatically fulfill the requirements, irrespective of what functionality it will support. To make these requirements meaningful we feel that they need to be complemented with some description of the functionality they are intended to support. 

One issue we have regards spatial audio capture and rendering. We feel that it is important to support the capture of several spatial audio sources in the same space (media stream and position in space) and that there is support for sending those spatial positions to the rendering instances. In the case where these spatial audio sources are associated with a video capture, it is also important to support the positioning of the audio sources in the x-y plain of the associated video window as this will make it possible to properly align the the rendered audio scene with the rendered video scene.

Another issue regards video capture of the same object from multiple capture angles and rendering of such video images on multiview screens. It is important for an endpoint with multiview screens to be able to obtain or evaluate the capture angles of the different video captures of the same object, so that it can display the different video images at the appropriate viewing angles.   

Below we have some suggestions on how requirements 1, 2 amd 3 could be complemented with relevant functionality support descriptions.

   REQMT-1:   The solution MUST support a means of describing the
              spatial arrangement of source video images sent in video
              streams in such a way that:
               - a client with a rendering layout that matches the particular 
                 capture layout is able to render the video images in a spatially
                 correct manner.
               - it is possible to make a spatially correct composite video image 
                 of several of the video images.
               - a client with multiview screens is able to obtain or evaluate the capture 
                 angles of video images that capture the same visual object from 
                 different angles.

   REQMT-2:   The solution MUST support a means of describing the
              spatial arrangement of source audio sounds sent in audio
              streams in such a way that: 
               - a client with an audio rendering layout that matches the 
                 particular audio capture layout is able to render the audio 
                 in a spatially correct manner.
               - it supports the spatial capture of several audio sources in the same space 
                 with a spatial position for each source.

   REQMT-3:   The solution MUST support a mechanism to allow spatial
              matching of audio and video streams from the same
              endpoints such that:
               - rendering clients are able to correctly position audio sources 
                 associated with a video image in the image plane of the rendered 
                 video window.
               - spatial audio sources associated with a video image can be positioned 
                 in the x-y plane of the associated video window.

Kind regards,

Erlendur Karlsson

---

From: "Gorzynski, Mark E (The Right One)" <[email-masked]>
Date: Thu, 12 May 2011 16:46:38 +0000

I think support for multi-view displays (multi-view, 3D, freeview, etc) should be optional not mandatory for CLUE.

I believe the requirements from Mr. Karlsson would demand a fairly 
extensive increase in complexity as they would require establishing 
spatial support such as camera viewing angle and
mapping spatial audio to video window position.

While this type of support is understandable and even desirable, I 
believe many systems would consider this as optional, not mandatory.

Mark.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Erlendur Karlsson
Sent: Wednesday, May 11, 2011 9:37 AM
To: Allyn Romanow (allyn)
Cc: CLUE
Subject: [clue] draft-romanow-clue-telepresence-requirements-02: Spatial Capture Description - Reqm 1, 2 and 3

Hi,

Kind regards,

Erlendur Karlsson
_______________________________________________
clue mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/clue

---

From: Stephan Wenger <[email-masked]>
Date: Thu, 12 May 2011 18:59:02 -0700

Hi Mark,
Are you sure that you are not mixing up requirements of specification with
requirements of implementation?
At this point, we are not necessarily discussing levels mandation of an
implementation.  I think it is totally OK to say that a "solution" (which
is: a future set of RFCs) MUST address a certain problem, but the
"solution" makes the support by products optional.
Stephan

On 5.12.2011 09:46 , "Gorzynski, Mark E (The Right One)"
<[email-masked]> wrote:

>I think support for multi-view displays (multi-view, 3D, freeview, etc)
>should be optional not mandatory for CLUE.
>
>I and some of my colleages have looked over the Spatial Capture
>Description Requirements (1,2 and 3) and we have some comments and
>suggestions on those.
>
>These requirements mandate means of describing spatial arrangements of
>video images and audio signals, but there is no mention of what
>particular functionality they should enable or support. Due to this any
>spatial arrangement description will automatically fulfill the
>requirements, irrespective of what functionality it will support. To make
>these requirements meaningful we feel that they need to be complemented
>with some description of the functionality they are intended to support.
>
>One issue we have regards spatial audio capture and rendering. We feel
>that it is important to support the capture of several spatial audio
>sources in the same space (media stream and position in space) and that
>there is support for sending those spatial positions to the rendering
>instances. In the case where these spatial audio sources are associated
>with a video capture, it is also important to support the positioning of
>the audio sources in the x-y plain of the associated video window as this
>will make it possible to properly align the the rendered audio scene with
>the rendered video scene.
>
>Another issue regards video capture of the same object from multiple
>capture angles and rendering of such video images on multiview screens.
>It is important for an endpoint with multiview screens to be able to
>obtain or evaluate the capture angles of the different video captures of
>the same object, so that it can display the different video images at the
>appropriate viewing angles.
>
>Below we have some suggestions on how requirements 1, 2 amd 3 could be
>complemented with relevant functionality support descriptions.
>
>

---

From: "Gorzynski, Mark E (The Right One)" <[email-masked]>
Date: Thu, 12 May 2011 17:03:49 +0000

Perhaps, good point.

I am concerned about the complexity of including specification support for multi-view.  My concern is about specification not implementation.

I ask us to be clear whether specification support for multi-view / 3D is critical now in CLUE.

Mark

-----Original Message-----
From: Stephan Wenger [mailto:[email-masked]] 
Sent: Thursday, May 12, 2011 6:59 PM
To: Gorzynski, Mark E (The Right One); CLUE
Subject: Re: [clue] draft-romanow-clue-telepresence-requirements-02: Spatial Capture Description - Reqm 1, 2 and 3

Hi Mark,
Are you sure that you are not mixing up requirements of specification with requirements of implementation?
At this point, we are not necessarily discussing levels mandation of an implementation.  I think it is totally OK to say that a "solution" (which
is: a future set of RFCs) MUST address a certain problem, but the "solution" makes the support by products optional.
Stephan

On 5.12.2011 09:46 , "Gorzynski, Mark E (The Right One)"
<[email-masked]> wrote:

>spatial support such as camera viewing angle and mapping spatial audio 
>to video window position.
>
>Subject: [clue] draft-romanow-clue-telepresence-requirements-02: 
>Spatial Capture Description - Reqm 1, 2 and 3
>
>Hi,
>
>requirements, irrespective of what functionality it will support. To 
>make these requirements meaningful we feel that they need to be 
>complemented with some description of the functionality they are intended to support.
>
>with a video capture, it is also important to support the positioning 
>of the audio sources in the x-y plain of the associated video window as 
>this will make it possible to properly align the the rendered audio 
>scene with the rendered video scene.
>
>obtain or evaluate the capture angles of the different video captures 
>of the same object, so that it can display the different video images 
>at the appropriate viewing angles.
>