Subject: Re: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

From: Rohit Abhishek <[email-masked]>
Date: Mon, 2 Nov 2020 16:18:09 +0000

Hi Christer,

Thanks for reading the draft and your reviews. I really appreciate your feedback. The questions here suggest more explanatory text about motivation is needed. I will address these in the next version.

From: Christer Holmberg <[email-masked]>
Date: Thursday, October 29, 2020 at 7:46 AM
To: Rohit Abhishek <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: RE: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Hi Rohit,

A few initial questions for clarifications:
RA: The assumption here is for an immersive telepresence session, one immersive video will be transmitted, whereas other streams will be transmitted as overlays. For ex, consider 10 people in a conference call; a user may stream an immersive video from one user, whereas 2D videos as overlays from other 8 participants (plus additional streams if any participant shares any additional media) . The total overlay streams may vary, for example, a session with a large number of audiences.

Q1:                      The draft references the telepresence use-case spec (RFC7205), but there is no text on why the actual CLUE mechanism isn’t feasible for what you want to do.
RA: I had referred to RFC7205 for definition purposes. CLUE enables interoperability between telepresence systems by exchanging information about the systems’ characteristics. Here, the cameras and screens may be arranged to provide a panoramic view of the remote room.  However, overlays were not considered in the CLUE mechanism.

Q2:                      It is unclear why you need to group the overlay streams together. If you only want to indicate that a stream is for overlay you can e.g., use an SDP attribute for that. You don’t need grouping.
RA: Of course, when a single or relatively fewer overlays are transmitted, grouping won’t be needed. However, when relatively larger numbers of users are in a session, it may be more favorable to use a grouping framework than using SDP attribute for each stream.

Q3:                      The draft talks about the overlay streams being associated with “immersive video”. But, there is no text about how this immersive video stream is represented, and how the overlay streams are associated with that immersive video stream.
RA: Thanks for the suggestion. I will update this in the next version

Q4:                      I assume the overlay streams can be multiplexed using the BUNDLE mechanism?
RA: There maybe multiple RTP sessions whose streams may have to be grouped. Using BUNDLE mechanism will only allow to group media streams within a single RTP session.  However, multiple media streams from various endpoints may be multiplexed into a single RTP session but would require RTP-level middle boxes. These middle boxes may add delay if a large number of streams are to be multiplexed, which even if very less, may not be favorable for telepresence sessions.

Best Regards,
Rohit

Regards,

Christer

From: mmusic <[email-masked]> On Behalf Of Rohit Abhishek
Sent: keskiviikko 28. lokakuuta 2020 0.23
To: [email-masked]
Subject: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Dear All,

I am currently working on an SDP extension facilitating overlays for immersive telepresence.

An individual draft has been posted at

https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt

Comments are welcome

Best Regards,

Rohit

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Tue, Oct 27, 2020 at 2:56 PM
Subject: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt
To: Rohit Abhishek <[email-masked]<mailto:[email-masked]>>

A new version of I-D, draft-abhishek-mmusic-overlay-grouping-00.txt
has been successfully submitted by Rohit Abhishek and posted to the
IETF repository.

Name:           draft-abhishek-mmusic-overlay-grouping
Revision:       00
Title:          SDP Overlay Grouping framework for immersive telepresence media streams
Document date:  2020-10-27
Group:          Individual Submission
Pages:          8
URL:            https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt
Status:         https://datatracker.ietf.org/doc/draft-abhishek-mmusic-overlay-grouping/
Htmlized:       https://datatracker.ietf.org/doc/html/draft-abhishek-mmusic-overlay-grouping
Htmlized:       https://tools.ietf.org/html/draft-abhishek-mmusic-overlay-grouping-00

Abstract:
   This document defines semantics that allow for signalling a new SDP
   group "OL" for overlays in an immersive telepresence session.  The
   "OL" attribute can be used by the application to relate all the
   overlay media streams enabling them to be added as overlay on top of
   the immersive video.  The overlay grouping semantics is required, if
   the media data is seperate and transported via different protocols.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

The IETF Secretariat

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 2 Nov 2020 23:41:57 +0000

Hi Rohit,

>RA: The assumption here is for an immersive telepresence session, one immersive video will be transmitted, whereas other streams will be transmitted as overlays. For ex, consider 10 people in a conference call; a
>user may stream an immersive video from one user, whereas 2D videos as overlays from other 8 participants (plus additional streams if any participant shares any additional media) . The total overlay streams may vary, for example, a session with a large number of audiences.
>
>Q1:                      The draft references the telepresence use-case spec (RFC7205), but there is no text on why the actual CLUE mechanism isn't feasible for what you want to do.
>RA: I had referred to RFC7205 for definition purposes. CLUE enables interoperability between telepresence systems by exchanging information about the systems' characteristics. Here, the cameras and screens may be arranged to provide a panoramic view of the remote room.  However, overlays were not considered in the CLUE mechanism.

So, spatial relationships etc are outside the scope of what you want to do? You simply want to indicate streams as overlay streams, but the receiving application will decide the order they are rendered in etc.

In addition, the user being able to switch between the main (immersive) video and an overlay is outside the scope?

I think the draft would need to be very clear on what is in the scope - and what isn't. Also, some text why the CLUE mechanism isn't used would be useful.

>Q2:                      It is unclear why you need to group the overlay streams together. If you only want to indicate that a stream is for overlay you can e.g., use an SDP attribute for that. You don't need grouping.
>RA: Of course, when a single or relatively fewer overlays are transmitted, grouping won't be needed. However, when relatively larger numbers of users are in a session, it may be more favorable to use a grouping framework than using SDP attribute for each stream.

Perhaps.

But, before deciding on a mechanism, it needs to be clear what the exact requirement is. And, if I understand correct, the requirement is to indicate that a video stream is an overlay stream?

>Q3:                      The draft talks about the overlay streams being associated with "immersive video". But, there is no text about how this immersive video stream is represented, and how the overlay streams are associated with that immersive video stream.
>RA: There maybe multiple RTP sessions whose streams may have to be grouped. Using BUNDLE mechanism will only allow to group media streams within a single RTP session.  However, multiple media streams from various endpoints may be
>multiplexed into a single RTP session but would require RTP-level middle boxes. These middle boxes may add delay if a large number of streams are to be multiplexed, which even if very less, may not be favorable for telepresence sessions.

I guess I was unclear. My suggestion was not to use BUNDLE instead of e.g., your suggested group. But, within such group, there is nothing that prevents overlay streams from being multiplexed?

Regards,

Christer

From: mmusic <[email-masked]<mailto:[email-masked]>> On Behalf Of Rohit Abhishek
Sent: keskiviikko 28. lokakuuta 2020 0.23
To: [email-masked]<mailto:[email-masked]>
Subject: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Dear All,

Comments are welcome

Best Regards,
Rohit

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Tue, Oct 27, 2020 at 2:56 PM
Subject: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt
To: Rohit Abhishek <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat

---

From: Rohit Abhishek <[email-masked]>
Date: Tue, 3 Nov 2020 14:41:41 +0000

Thanks again Christer for your comments/feedbacks! :-)

Hi Rohit,

I think the draft would need to be very clear on what is in the scope – and what isn’t. Also, some text why the CLUE mechanism isn’t used would be useful.

Yes, spatial relationships are currently outside the scope of the proposal.

My apologies for not being clear enough wrt to switching between immersive video and overlays. I do have some related images which I am planning to include in the next version which would definitely bring clarity to what is being proposed here.

Perhaps.

But, before deciding on a mechanism, it needs to be clear what the exact requirement is. And, if I understand correct, the requirement is to indicate that a video stream is an overlay stream?
Yes, the requirement is to indicate that a stream is an overlay stream. I will update the exact requirements in the next version.

I guess I was unclear. My suggestion was not to use BUNDLE instead of e.g., your suggested group. But, within such group, there is nothing that prevents overlay streams from being multiplexed?
My understanding is there are limitations which may prevent overlay streams from being multiplexed. Please allow me to investigate more and address this in the next version.

Best Regards,
Rohit

Regards,

Christer

Dear All,

Comments are welcome

Best Regards,
Rohit

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Tue, Oct 27, 2020 at 2:56 PM
Subject: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt
To: Rohit Abhishek <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 3 Nov 2020 10:32:23 -0500

Rohit,

IMO a lot more content is needed to clarify what you are proposing.

For one thing, I gather this draft only covers signaling for the overlay 
streams, not for the immersive video stream. I find almost no 
information on how the immersive video is signaled. (I see the reference 
to [ISO23090], but apparently I have to pay to get a copy of that.) ISTM 
that it is pretty important to associate the overlays with the immersive 
stream in the signaling.

You apparently are aware of CLUE. It didn't contemplate exactly the case 
you are interested in, but it is clearly very closely related, in that 
it has mechanisms to associate multiple streams together as parts of a 
telepresence session and to describe their relationship to one another.

I think it would be straightforward to extend CLUE to cover your 
immersive telepresence. (It might take some work to incorporate the 
geometry / coordinate system of the immersive stream into the CLUE 
framework.)

Of course CLUE hasn't been an overwhelming success. :-(
But if there is a new interest in *immersive* telepresence then perhaps 
there is some motivation to reopen the work.

If you intend this as an *alternative* to CLUE for telepresence, then 
you need a lot more info in this document, or an associated framework 
document, to lay out the complete environment in which this overlay 
piece resides. Perhaps that is all defined in ISO documents. If so then 
it will be difficult to do this work in IETF without access to the 
relevant ISO documents.

	Thanks,
	Paul

On 11/2/20 11:18 AM, Rohit Abhishek wrote:
> Hi Christer,
> 
> Thanks for reading the draft and your reviews. I really appreciate your 
> feedback. The questions here suggest more explanatory text about 
> motivation is needed. I will address these in the next version.
> 
> *From: *Christer Holmberg <[email-masked]>
> *Date: *Thursday, October 29, 2020 at 7:46 AM
> *To: *Rohit Abhishek <[email-masked]>, "[email-masked]" 
> <[email-masked]>
> *Subject: *RE: [MMUSIC] Fwd: New Version Notification for 
> draft-abhishek-mmusic-overlay-grouping-00.txt
> 
> Hi Rohit,
> 
> *RA: The assumption here is for an immersive telepresence session, one 
> immersive video will be transmitted, whereas other streams will be 
> transmitted as overlays. For ex, consider 10 people in a conference 
> call; a user may stream an immersive video from one user, whereas 2D 
> videos as overlays from other 8 participants (plus additional streams if 
> any participant shares any additional media) . The total overlay streams 
> may vary, for example, a session with a large number of audiences.*
> 
> Q1:                      The draft references the telepresence use-case 
> spec (RFC7205), but there is no text on why the actual CLUE mechanism 
> isn’t feasible for what you want to do.
> 
> *RA: I had referred to RFC7205 for definition purposes. CLUE enables 
> interoperability between telepresence systems by exchanging information 
> about the systems’ characteristics. Here, the cameras and screens may be 
> arranged to provide a panoramic view of the remote room.  However, 
> overlays were not considered in the CLUE mechanism.*
> 
> Q2:                      It is unclear why you need to group the overlay 
> streams together. If you only want to indicate that a stream is for 
> overlay you can e.g., use an SDP attribute for that. You don’t need 
> grouping.
> 
> *RA: Of course, when a single or relatively fewer overlays are 
> transmitted, grouping won’t be needed. However, when relatively larger 
> numbers of users are in a session, it may be more favorable to use a 
> grouping framework than using SDP attribute for each stream.*
> 
> Q3:                      The draft talks about the overlay streams being 
> associated with “immersive video”. But, there is no text about how this 
> immersive video stream is represented, and how the overlay streams are 
> associated with that immersive video stream.
> 
> *RA: Thanks for the suggestion. I will update this in the next version*
> 
> *RA: There maybe multiple RTP sessions whose streams may have to be 
> grouped. Using BUNDLE mechanism will only allow to group media streams 
> within a single RTP session.  However, multiple media streams from 
> various endpoints may be multiplexed into a single RTP session but would 
> require RTP-level middle boxes. These middle boxes may add delay if a 
> large number of streams are to be multiplexed, which even if very less, 
> may not be favorable for telepresence sessions.*
> 
> *Best Regards,*
> 
> *Rohit*
> 
> Regards,
> 
> Christer
> 
> *From:* mmusic <[email-masked]> *On Behalf Of *Rohit Abhishek
> *Sent:* keskiviikko 28. lokakuuta 2020 0.23
> *To:* [email-masked]
> *Subject:* [MMUSIC] Fwd: New Version Notification for 
> draft-abhishek-mmusic-overlay-grouping-00.txt
> 
> Dear All,
> 
> <https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt>
> 
> Comments are welcome
> 
> Best Regards,
> 
> Rohit
> 
> From: <[email-masked] <mailto:[email-masked]>>
> Subject: New Version Notification for 
> draft-abhishek-mmusic-overlay-grouping-00.txt
> URL: 
> <https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt>
> Status: 
> https://datatracker.ietf.org/doc/draft-abhishek-mmusic-overlay-grouping/ 
> <https://datatracker.ietf.org/doc/draft-abhishek-mmusic-overlay-grouping/>
> Htmlized: 
> https://datatracker.ietf.org/doc/html/draft-abhishek-mmusic-overlay-grouping 
> <https://datatracker.ietf.org/doc/html/draft-abhishek-mmusic-overlay-grouping>
> Htmlized: 
> https://tools.ietf.org/html/draft-abhishek-mmusic-overlay-grouping-00 
> <https://tools.ietf.org/html/draft-abhishek-mmusic-overlay-grouping-00>
> 
> 
> Abstract:
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>

---

From: Stephan Wenger <[email-masked]>
Date: Tue, 10 Nov 2020 16:50:22 +0000

Hi Paul,

I'm not Rohit, but we work closely together.  Please see inline in blue.

Thanks,

Stephan

On 11/3/20, 07:32, "mmusic on behalf of Paul Kyzivat" <[email-masked] on behalf of [email-masked]> wrote:

    Rohit,

Agreed.

    streams, not for the immersive video stream.

The draft covers the relationship between a “base” or “background” stream (which could be immersive, see below), and one or more overlays.

I find almost no

    to [ISO23090], but apparently I have to pay to get a copy of that.)

Forget that reference for now.  The background of this draft relates to work in 3GPP SA4, specifically ITT4RT (Support of Immersive Teleconferencing and Telepresence for Remote Terminals<https://portal.3gpp.org/desktopmodules/WorkItem/WorkItemDetails.aspx?workitemId=820003>).  They talk a lot about the background being immersive, projected video, but for our purposes here that background could also be a simple planar video stream.

ISTM

    stream in the signaling.

Beyond the information that an overlay is to be displayed “on top” of whatever the background may be?  I’m not sure that is necessarily correct. Making an oversimplifying analogy, when you draw something in your favorite drawing program, you can send objects “to the front” or  “to the back”.  That’s all the functionality we are currently after (though we are considering adding, to this draft, information that would establish a hierarchy of overlays beyond their relationship with the background, and also some geometric information.

    framework.)

Yes, we looked at CLUE.  CLUE could probably be extended to support such use cases, but it would be a pretty heavy operation on a protocol suite that struggles to find acceptance outside the narrowly defined, high-end environment it was designed for.  Specifically, the issue at hand is that CLUE describes the geometric relationship of media sources (camera positions, for example), which can be expressed as a point of origin and some angles.  Those areas of capture can, and typically do, overlap at least to some extent.  No foreground/background relationship is defined in CLUE between these sources, nor is transparency (which may be something to look into for our draft).  Our equivalent of CLUE sources are video streams—planes of reconstructed samples.  Their relationship with the background is undefined.  Where those planes are rendered is a user interface decision in the receiver.  We are not trying to re-invent scene descriptions here.

https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_110-e/Docs/S4-201194.zip covers the current thinking in 3GPP (phase 1—requirements phase), and the figs in section 3.8 may be helpful.

    relevant ISO documents.

Agreed, but I hope I managed to describe above why an extension to the CLUE protocol suite would be a lot heavier work than something as simple as described in the draft at hand—using a reference to a document that’s available to everyone.

                Thanks,

                Paul

    > Q4:                      I assume the overlay streams can be multiplexed

    > using the BUNDLE mechanism?

    > I am currently working on an SDP extension facilitating overlays for

    > immersive telepresence.

    > To: Rohit Abhishek <[email-masked]

    > <mailto:[email-masked]>>

    > Title:          SDP Overlay Grouping framework for immersive

    > telepresence media streams

    > until the htmlized version and diff are available at tools.ietf.org

    > <http://tools.ietf.org>.

    mmusic mailing list

    [email-masked]

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 10 Nov 2020 12:39:59 -0500

Stephan,

On 11/10/20 11:50 AM, Stephan Wenger wrote:
> On 11/3/20, 07:32, "mmusic on behalf of Paul Kyzivat" 
> <[email-masked] on behalf of [email-masked]> wrote:
> 
>      Rohit,
> 
> The draft covers the relationship between a “base” or “background” 
> stream (which could be immersive, see below), and one or more overlays.
> 
> I find almost no
> 
> Forget that reference for now.  The background of this draft relates to 
> work in 3GPP SA4, specifically ITT4RT (*Support of Immersive 
> Teleconferencing and Telepresence for Remote Terminals 
> <https://portal.3gpp.org/desktopmodules/WorkItem/WorkItemDetails.aspx?workitemId=820003>*).  
> They talk a lot about the background being immersive, projected video, 
> but for our purposes here that background could also be a simple planar 
> video stream.
> 
> ISTM
> 
> Beyond the information that an overlay is to be displayed “on top” of 
> whatever the background may be?  I’m not sure that is necessarily 
> correct. Making an oversimplifying analogy, when you draw something in 
> your favorite drawing program, you can send objects “to the front” or  
> “to the back”.  That’s all the functionality we are currently after 
> (though we are considering adding, to this draft, information that would 
> establish a hierarchy of overlays beyond their relationship with the 
> background, and also some geometric information.

My concern is that by not explicitly associating the overlay with the 
background you are assuming that there *is* a single well defined 
background that both ends agree on. What if there are multiple video 
streams in the session other than the overlay?

> Yes, we looked at CLUE.  CLUE could probably be extended to support such 
> use cases, but it would be a pretty heavy operation on a protocol suite 
> that struggles to find acceptance outside the narrowly defined, high-end 
> environment it was designed for.  Specifically, the issue at hand is 
> that CLUE describes the geometric relationship of media sources (camera 
> positions, for example), which can be expressed as a point of origin and 
> some angles.  Those areas of capture can, and typically do, overlap at 
> least to some extent.  No foreground/background relationship is defined 
> in CLUE between these sources, nor is transparency (which may be 
> something to look into for our draft). 

Yes. Clearly it would require enhancements to the clue model.

> Our equivalent of CLUE sources 
> are video streams—planes of reconstructed samples.  Their relationship 
> with the background is undefined.  Where those planes are rendered is a 
> user interface decision in the receiver.  We are not trying to re-invent 
> scene descriptions here.
> 
> https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_110-e/Docs/S4-201194.zip 
> <https://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_110-e/Docs/S4-201194.zip> 
> covers the current thinking in 3GPP (phase 1—requirements phase), and 
> the figs in section 3.8 may be helpful.
> 
> Agreed, but I hope I managed to describe above why an extension to the 
> CLUE protocol suite would be a lot heavier work than something as simple 
> as described in the draft at hand—using a reference to a document that’s 
> available to everyone.

Without getting into the weeds I can accept your conclusion that CLUE 
isn't appropriate here. (It might be good to put a few words saying why 
CLUE isn't a suitable base for this.)

I only ask that you resolve ambiguities in the document.

	Thanks,
	Paul

>      > Hi Christer,
>      >
>      > Thanks for reading the draft and your reviews. I really 
> appreciate your
> 
>      > feedback. The questions here suggest more explanatory text about
> 
>      > motivation is needed. I will address these in the next version.
> 
>      >
> 
>      > *From: *Christer Holmberg <[email-masked]>
> 
>      > *Date: *Thursday, October 29, 2020 at 7:46 AM
> 
>      > *To: *Rohit Abhishek <[email-masked]>, "[email-masked]"
> 
>      > <[email-masked]>
> 
>      > *Subject: *RE: [MMUSIC] Fwd: New Version Notification for
> 
>      > draft-abhishek-mmusic-overlay-grouping-00.txt
> 
>      >
> 
>      > Hi Rohit,
> 
>      >
> 
>      > A few initial questions for clarifications:
> 
>      >
> 
>      > *RA: The assumption here is for an immersive telepresence 
> session, one
> 
>      > immersive video will be transmitted, whereas other streams will be
> 
>      > transmitted as overlays. For ex, consider 10 people in a conference
> 
>      > call; a user may stream an immersive video from one user, whereas 2D
> 
>      > videos as overlays from other 8 participants (plus additional 
> streams if
> 
>      > any participant shares any additional media) . The total overlay 
> streams
> 
>      > may vary, for example, a session with a large number of audiences.*
> 
>      >
> 
>      > Q1:                      The draft references the telepresence 
> use-case
> 
>      > spec (RFC7205), but there is no text on why the actual CLUE 
> mechanism
> 
>      > isn’t feasible for what you want to do.
> 
>      >
> 
>      > *RA: I had referred to RFC7205 for definition purposes. CLUE enables
> 
>      > interoperability between telepresence systems by exchanging 
> information
> 
>      > about the systems’ characteristics. Here, the cameras and screens 
> may be
> 
>      > arranged to provide a panoramic view of the remote room.  However,
> 
>      > overlays were not considered in the CLUE mechanism.*
> 
>      >
> 
>      > Q2:                      It is unclear why you need to group the 
> overlay
> 
>      > streams together. If you only want to indicate that a stream is for
> 
>      > overlay you can e.g., use an SDP attribute for that. You don’t need
> 
>      > grouping.
> 
>      >
> 
>      > *RA: Of course, when a single or relatively fewer overlays are
> 
>      > transmitted, grouping won’t be needed. However, when relatively 
> larger
> 
>      > numbers of users are in a session, it may be more favorable to use a
> 
>      > grouping framework than using SDP attribute for each stream.*
> 
>      >
> 
>      > Q3:                      The draft talks about the overlay 
> streams being
> 
>      > associated with “immersive video”. But, there is no text about 
> how this
> 
>      > immersive video stream is represented, and how the overlay 
> streams are
> 
>      > associated with that immersive video stream.
> 
>      >
> 
>      > *RA: Thanks for the suggestion. I will update this in the next 
> version*
> 
>      >
> 
>      > Q4:                      I assume the overlay streams can be 
> multiplexed
> 
>      > using the BUNDLE mechanism?
> 
>      >
> 
>      > *RA: There maybe multiple RTP sessions whose streams may have to be
> 
>      > grouped. Using BUNDLE mechanism will only allow to group media 
> streams
> 
>      > within a single RTP session.  However, multiple media streams from
> 
>      > various endpoints may be multiplexed into a single RTP session 
> but would
> 
>      > require RTP-level middle boxes. These middle boxes may add delay 
> if a
> 
>      > large number of streams are to be multiplexed, which even if very 
> less,
> 
>      > may not be favorable for telepresence sessions.*
> 
>      >
> 
>      > *Best Regards,*
> 
>      >
> 
>      > *Rohit*
> 
>      >
> 
>      > Regards,
> 
>      >
> 
>      > Christer
> 
>      >
> 
>      > *From:* mmusic <[email-masked]> *On Behalf Of *Rohit 
> Abhishek
> 
>      > *Sent:* keskiviikko 28. lokakuuta 2020 0.23
> 
>      > *To:* [email-masked]
> 
>      > *Subject:* [MMUSIC] Fwd: New Version Notification for
> 
>      > draft-abhishek-mmusic-overlay-grouping-00.txt
> 
>      >
> 
>      > Dear All,
> 
>      >
> 
>      > I am currently working on an SDP extension facilitating overlays for
> 
>      > immersive telepresence.
> 
>      >
> 
>      > An individual draft has been posted at
> 
>      >
> 
>      > 
>      > 
>      >
>      > Comments are welcome
>      >
>      > Best Regards,
>      >
>      > Rohit
>      >
>      > ---------- Forwarded message ---------
> 
>      > From: <[email-masked] <mailto:[email-masked]>>
> 
>      > Date: Tue, Oct 27, 2020 at 2:56 PM
> 
>      > Subject: New Version Notification for
> 
>      > draft-abhishek-mmusic-overlay-grouping-00.txt
> 
>      > To: Rohit Abhishek <[email-masked]
> 
>      > <mailto:[email-masked]>>
> 
>      >
> 
>      >
> 
>      >
> 
>      >
> 
>      > A new version of I-D, draft-abhishek-mmusic-overlay-grouping-00.txt
> 
>      > has been successfully submitted by Rohit Abhishek and posted to the
> 
>      > IETF repository.
> 
>      >
> 
>      > Name:           draft-abhishek-mmusic-overlay-grouping
> 
>      > Revision:       00
> 
>      > Title:          SDP Overlay Grouping framework for immersive
> 
>      > telepresence media streams
> 
>      > Document date:  2020-10-27
> 
>      > Group:          Individual Submission
> 
>      > Pages:          8
> 
>      > URL:
> 
>      > 
>      > 
>      > Status:
>      > 
>      > 
>      > Htmlized:
>      > 
>      > 
>      > Htmlized:
>      > 
>      > 
>      >
>      >
>      > Abstract:
>      >     This document defines semantics that allow for signalling a 
> new SDP
> 
>      >     group "OL" for overlays in an immersive telepresence 
> session.  The
> 
>      >     "OL" attribute can be used by the application to relate all the
> 
>      >     overlay media streams enabling them to be added as overlay on 
> top of
> 
>      >     the immersive video.  The overlay grouping semantics is 
> required, if
> 
>      >     the media data is seperate and transported via different 
> protocols.
> 
>      >
> 
>      >
> 
>      >
> 
>      >
> 
>      > Please note that it may take a couple of minutes from the time of 
> submission
> 
>      > until the htmlized version and diff are available at tools.ietf.org
> 
>      > <http://tools.ietf.org>.
> 
>      >
> 
>      > The IETF Secretariat
> 
>      >
> 
>      >
> 
>      > _______________________________________________
> 
>      > mmusic mailing list
> 
>      > [email-masked]
> 
>      > https://www.ietf.org/mailman/listinfo/mmusic
> 
>      >
>

---

From: Christer Holmberg <[email-masked]>
Date: Thu, 12 Nov 2020 18:41:39 +0000

Hi,

Based on my understanding, and what I have heard and read, I would also agree that CLUE is not needed. However, as Paul also suggested, it would be good to add some text about that.

I also agree that it must explicitly be possible to associate the overlay streams with a given "main stream". We shall not assume that there will only be one main stream, eventhough that may be the case 3GPP is currently looking into.

If we want to use the grouping mechanism, one way would be to place the main stream and the overlay stream into the same group, and then e.g., use an attribute to indicate which is main and which is overlay. But again, let's get the requirements clear before we start designing the protocol mechanism :)

Regards,

Christer

-----Original Message-----
From: mmusic <[email-masked]> On Behalf Of Paul Kyzivat
Sent: tiistai 10. marraskuuta 2020 19.40
To: Stephan Wenger <[email-masked]>; [email-masked]
Subject: Re: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Stephan,

On 11/10/20 11:50 AM, Stephan Wenger wrote:
> Forget that reference for now.  The background of this draft relates 
> to work in 3GPP SA4, specifically ITT4RT (*Support of Immersive 
> <https://protect2.fireeye.com/v1/url?k=11f71834-4e6c2105-11f758af-86e2237f51fb-2bcb4dbf2fce02d0&q=1&e=230e16d8-da72-481d-8298-2d6153cfc91e&u=https%3A%2F%2Fportal.3gpp.org%2Fdesktopmodules%2FWorkItem%2FWorkItemDetails.aspx%3FworkitemId%3D820003>*).
> but for our purposes here that background could also be a simple 
> planar video stream.
> 
> ISTM
> 
> (though we are considering adding, to this draft, information that 
> would establish a hierarchy of overlays beyond their relationship with 
> the background, and also some geometric information.

My concern is that by not explicitly associating the overlay with the background you are assuming that there *is* a single well defined background that both ends agree on. What if there are multiple video streams in the session other than the overlay?

> Yes, we looked at CLUE.  CLUE could probably be extended to support 
> such use cases, but it would be a pretty heavy operation on a protocol 
> suite that struggles to find acceptance outside the narrowly defined, 
> high-end environment it was designed for.  Specifically, the issue at 
> hand is that CLUE describes the geometric relationship of media 
> sources (camera positions, for example), which can be expressed as a 
> point of origin and some angles.  Those areas of capture can, and 
> typically do, overlap at least to some extent.  No 
> foreground/background relationship is defined in CLUE between these 
> sources, nor is transparency (which may be something to look into for our draft).

> with the background is undefined.  Where those planes are rendered is 
> a user interface decision in the receiver.  We are not trying to 
> re-invent scene descriptions here.
> 
> https://protect2.fireeye.com/v1/url?k=62e03972-3d7b0043-62e079e9-86e22
> 37f51fb-28c26b67238f4c8f&q=1&e=230e16d8-da72-481d-8298-2d6153cfc91e&u=
> https%3A%2F%2Fwww.3gpp.org%2Fftp%2Ftsg_sa%2FWG4_CODEC%2FTSGS4_110-e%2F
> Docs%2FS4-201194.zip 
> <https://protect2.fireeye.com/v1/url?k=cbce09a5-94553094-cbce493e-86e2
> 237f51fb-b2974a8cc1dbc52f&q=1&e=230e16d8-da72-481d-8298-2d6153cfc91e&u
> =https%3A%2F%2Fwww.3gpp.org%2Fftp%2Ftsg_sa%2FWG4_CODEC%2FTSGS4_110-e%2
> FDocs%2FS4-201194.zip> covers the current thinking in 3GPP (phase 
> 1—requirements phase), and the figs in section 3.8 may be helpful.
> 
> CLUE protocol suite would be a lot heavier work than something as 
> simple as described in the draft at hand—using a reference to a 
> document that’s available to everyone.

Without getting into the weeds I can accept your conclusion that CLUE isn't appropriate here. (It might be good to put a few words saying why CLUE isn't a suitable base for this.)

	Thanks,
	Paul

>      > any participant shares any additional media) . The total 
> overlay streams
> 
>      > about the systems’ characteristics. Here, the cameras and 
> screens may be
> 
>      > Q2:                      It is unclear why you need to group 
> the overlay
> 
>      > require RTP-level middle boxes. These middle boxes may add 
> delay if a
> 
>      > large number of streams are to be multiplexed, which even if 
> very less,
> 
>      > A new version of I-D, 
> <https://tools.ietf.org/html/draft-abhishek-mmusic-overlay-grouping-00
> >
> 
>      >
> 
>      >
> 
>      > Abstract:
> 
>      >     overlay media streams enabling them to be added as overlay 
> on top of
> 
>      > Please note that it may take a couple of minutes from the time 
> of submission
>

---

From: "Roni Even" <[email-masked]>
Date: Tue, 17 Nov 2020 20:39:42 +0200

Hi,

Read the draft and the discussion thread from Christer, Paul and Stephan.

I am also not clear about the use case here. Is there one main stream and an overlay that covers all of it? 

I am not sure how the SDP example of one video and one audio steam has to do with the case described and why you need an OL group.

Stephan gave the example of drawing program about back and front but those programs also talk about transparency between the layers. For example can an overlay be semitransparent over the main?

Roni Even 

From: mmusic [mailto:[email-masked]] On Behalf Of Rohit Abhishek
Sent: Wednesday, October 28, 2020 12:23 AM
To: [email-masked]
Subject: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Dear All,

 <https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt> https://www.ietf.org/archive/id/draft-abhishek-mmusic-overlay-grouping-00.txt

Comments are welcome

Best Regards,

Rohit

---------- Forwarded message ---------
From: <[email-masked]>
Date: Tue, Oct 27, 2020 at 2:56 PM
Subject: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt
To: Rohit Abhishek <[email-masked]>

Abstract:

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Rohit Abhishek <[email-masked]>
Date: Tue, 17 Nov 2020 19:01:26 +0000

Hi Roni,

Thanks for reading the draft and your comments/concerns.

I am currently working on the next version and addressing the concerns raised by Christer, Paul + You.  Please allow me sometime to upload the next version.

Best Regards,

Rohit

From: Roni Even <[email-masked]>
Date: Tuesday, November 17, 2020 at 10:39 AM
To: 'Rohit Abhishek' <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: RE: [MMUSIC] Fwd: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt

Hi,

Roni Even

Dear All,

Comments are welcome

Best Regards,

Rohit

---------- Forwarded message ---------
From: <[email-masked]<mailto:[email-masked]>>
Date: Tue, Oct 27, 2020 at 2:56 PM
Subject: New Version Notification for draft-abhishek-mmusic-overlay-grouping-00.txt
To: Rohit Abhishek <[email-masked]<mailto:[email-masked]>>

Abstract:

The IETF Secretariat