Subject: Re: [MMUSIC] New Version Notification for draft-abhishek-mmusic-superimposition-grouping-00.txt(Internet mail) - order and transparency

From: Christer Holmberg <[email-masked]>
Date: Tue, 19 Jan 2021 18:39:28 +0000

Hi,

...

>>    Finally, ISTM that you don't need to treat the background document as a 
>>    special case. The grouping mechanism can be used to identify all the 
>>    media streams participating in this superposition mechanism, including 
>>    the one you are considering to be the background.
>>
>>    IIUC the scheme you describe defines the result of superposition based 
>>    on the transparency values of the individual streams, so that no other 
>>    ordering mechanism is needed. (Do I have that right?) If so, isn't the 
>>    background just the media stream with tranparency = 100?
>
> Rohit: The ordering mechanism will be needed. Suppose we have a background X with transparency=100, and a superimposed media A with transparency=50. 
> A over X would mean that X would visible partially through A, however X over A would mean that A wouldn't show through X.

That is my understanding too, after I started to look a little bit more into this: for each stream within the group, we need the order value AND the transparency value.

Regards,

Christer

    > Best Regards,
    > Rohit
    > 
    > On 1/14/21, 9:25 AM, "mmusic on behalf of Paul Kyzivat" <mailto:[email-masked] on behalf of [email-masked]> wrote:
    > 
    >      On 1/14/21 4:54 AM, Christer Holmberg wrote:
    >      > Hi,
    >      >
    >      >> One more thing, regarding transparency:
    >      >>
    >      >> I'm fuzzy on how this is intended to work. I would think that each superposition layer would have some parts that are fully transparent.
    >      >> (Transparent pixels.) And then the transparency attribute would apply to the parts that aren't fully transparent. But that is only a guess. I think you need to say more about how the superposition is supposed to work.
    >      >
    >      > Isn't "transparent background" information carried within the video stream (codec or container) itself?
    > 
    >      That is my impression, but I don't really know. But the transparency
    >      attribute must do *something*.
    > 
    >      The draft needs to explain better.
    > 
    >        Thanks,
    >        Paul
    > 
    >      > Regards,
    >      >
    >      > Christer
    >      >
    >      >
    >      >
    >      > On 12/14/20 11:21 AM, Paul Kyzivat wrote:
    >      >> Rohit,
    >      >>
    >      >> Here are a few more points I forgot:
    >      >>
    >      >> 1) syntax
    >      >>
    >      >> In section 3 you repeat the syntax for the 'mid' attribute from RFC5888.
    >      >> I guess you don't intend it to be normative, but it gave me that
    >      >> impression.
    >      >>
    >      >> I don't think you need to repeat the syntax here. It should be
    >      >> sufficient to mention that you follow RFC5888 in use of the 'mid'
    >      >> attribute to identify the media sections that are to be included in
    >      >> the superposition.
    >      >>
    >      >> Similarly in section 5, I suggest that you only provide ABNF for the
    >      >> additions to syntax required for your extension. Specifically:
    >      >>
    >      >>     semantics =/ "superposition" <semantics as defined in RFC5888>
    >      >>
    >      >> (Everything else is inherited from RFC5888.)
    >      >>
    >      >> 2) applicability
    >      >>
    >      >> Regarding the following in section 3:
    >      >>
    >      >>      ... An application that receives a session
    >      >>      description that contains "m" lines grouped together using "S"
    >      >>      semantics MUST superimpose the corresponding media streams on top
    >      >> of
    >      >>      the background media stream.
    >      >>
    >      >> You need to recognize that because this feature is an extension its
    >      >> requirements can only apply to those that implement the extension. Its
    >      >> best to call that out specifically.
    >      >>
    >      >> Also, please give some thought to what will happen a device supporting
    >      >> this negotiates a session with a device that doesn't. The
    >      >> non-supporting device will treat these as independent media streams.
    >      >> If that is a problem for you then we need to discuss strategies for
    >      >> avoiding that situation.
    >      >>
    >      >> 3) differing size/resolution
    >      >>
    >      >> How the overlaying will work seems well defined if the image width and
    >      >> height in pixels and pixel shape is the same for all the media streams
    >      >> in the superposition group. But what if they aren't?
    >      >>
    >      >> I think the document should specify what should happen in such cases.
    >      >>
    >      >>       Thanks,
    >      >>       Paul
    >      >>
    >      >> On 12/13/20 3:54 PM, Paul Kyzivat wrote:
    >      >>> On 12/12/20 2:06 PM, Christer Holmberg wrote:
    >      >>>> Hi,
    >      >>>>
    >      >>>> A few comments (some of which I gave already previously):
    >      >>>
    >      >>> Christer covered most of what I wanted to say. I'll just add a few
    >      >>> tweaks below.
    >      >>>
    >      >>>> Q1:
    >      >>>>
    >      >>>> In Section 1, where you talk about the limitations of the mechanism,
    >      >>>> you should mention that CLUE must be used something more "fancy" is
    >      >>>> needed.
    >      >>>>
    >      >>>> ---
    >      >>>>
    >      >>>> Q2:
    >      >>>>
    >      >>>> In Section 3, you require a specific order of the m- lines in the
    >      >>>> SDP. I think SDP explicitly says that the order of m- lines is not
    >      >>>> relevant. Also, I know that parsers may change the orders of the m-
    >      >>>> lines.
    >      >>>>
    >      >>>> (I also think that any counting should start from 1, not 0)
    >      >>>>
    >      >>>> My suggestion would be to have an explicit "order" attribute instead.
    >      >>>
    >      >>> I think it would be acceptable to use the order of the tokens in the
    >      >>> a:group line to define the ordering of the "layers". The "lowest"
    >      >>> layer would then be the background on which the others are overlain.
    >      >>> (Whether the lowest layer is the first or last in the list is TBD.)
    >      >>>
    >      >>>> ---
    >      >>>>
    >      >>>> Q3:
    >      >>>>
    >      >>>> The example in Section 6 talks about "a background video". But, this
    >      >>>> background video is not described in the SDP.
    >      >>>>
    >      >>>> Related to that, if the SDP describes multiple "background videos",
    >      >>>> you need to describe how you map superimposed videos to a specific
    >      >>>> background video.
    >      >>>>
    >      >>>> Based on the text in Section 3 (see Q2), I am not sure whether the
    >      >>>> 0th m- line is supposed to be the background video, but the text in
    >      >>>> Section 6 says that both m- lines are for superimposed video streams.
    >      >>>
    >      >>> See my comment above.
    >      >>>
    >      >>> The background layer would presumably have a transparency of zero.
    >      >>>
    >      >>> (It would also be good to specify that if the transparency attribute
    >      >>> is omitted it defaults to zero.)
    >      >>>
    >      >>>> ---
    >      >>>>
    >      >>>> Q4:
    >      >>>>
    >      >>>> What happens to the superimposed videos if I disable/remove/mute the
    >      >>>> background video?
    >      >>>>
    >      >>>> ---
    >      >>>>
    >      >>>> Q5:
    >      >>>>
    >      >>>> This is editorial, but please call the group attribute something
    >      >>>> else than just "S". "supim", "spim" or something :)
    >      >>>
    >      >>> Yes please. "S" is way to cryptic.
    >      >>>
    >      >>>       Thanks,
    >      >>>       Paul
    >      >>>
    >      >>>> Regards,
    >      >>>>
    >      >>>> Christer
    >      >>>>
    >      >>>>
    >      >>>>
    >      >>>> -----Original Message-----
    >      >>>> From: mmusic <mailto:[email-masked]> On Behalf Of Rohit Abhishek
    >      >>>> Sent: lauantai 12. joulukuuta 2020 0.28
    >      >>>> To: mailto:[email-masked]
    >      >>>> Subject: [MMUSIC] FW: New Version Notification for
    >      >>>> draft-abhishek-mmusic-superimposition-grouping-00.txt
    >      >>>>
    >      >>>> Dear All,
    >      >>>>
    >      >>>> A new version of the draft “SDP Superimposition Grouping framework”
    >      >>>> has been uploaded.
    >      >>>> We have changed the name from  “SDP Overlay Grouping framework for
    >      >>>> immersive telepresence media streams” to  “SDP Superimposition
    >      >>>> Grouping framework” as the previous name wasn’t an accurate
    >      >>>> description of the content. Therefore, we had to start from version 00.
    >      >>>>
    >      >>>> Thank you all for the comments/suggestions on the previous draft.
    >      >>>> We have tried to address them in this version. Specifically, a
    >      >>>> transparency attribute and an informative Section on relationship
    >      >>>> with CLUE has been added.
    >      >>>>
    >      >>>> Comments/Questions/Feedbacks are welcome.
    >      >>>>
    >      >>>> Best Regards,
    >      >>>> Rohit
    >      >>>>
    >      >>>> On 12/11/20, 12:41 PM, "mailto:[email-masked]"
    >      >>>> <mailto:[email-masked]> wrote:
    >      >>>>
    >      >>>>
    >      >>>>       A new version of I-D,
    >      >>>> draft-abhishek-mmusic-superimposition-grouping-00.txt
    >      >>>>       has been successfully submitted by Rohit Abhishek and posted to
    >      >>>> the
    >      >>>>       IETF repository.
    >      >>>>
    >      >>>>       Name:        draft-abhishek-mmusic-superimposition-grouping
    >      >>>>       Revision:    00
    >      >>>>       Title:        SDP Superimposition Grouping framework
    >      >>>>       Document date:    2020-12-11
    >      >>>>       Group:        Individual Submission
    >      >>>>       Pages:        8
    >      >>>>       URL:
    >      >>>> https://www.ietf.org/archive/id/draft-abhishek-mmusic-superimpositio
    >      >>>> n-grouping-00.txt
    >      >>>>
    >      >>>>       Status:
    >      >>>> https://datatracker.ietf.org/doc/draft-abhishek-mmusic-superimpositi
    >      >>>> on-grouping/
    >      >>>>
    >      >>>>       Htmlized:
    >      >>>> https://datatracker.ietf.org/doc/html/draft-abhishek-mmusic-superimp
    >      >>>> osition-grouping
    >      >>>>
    >      >>>>       Htmlized:
    >      >>>> https://tools.ietf.org/html/draft-abhishek-mmusic-superimposition-gr
    >      >>>> ouping-00
    >      >>>>
    >      >>>>
    >      >>>>
    >      >>>>       Abstract:
    >      >>>>          This document defines semantics that allow for signaling a
    >      >>>> new SDP
    >      >>>>          group "S" for superimposed media in an SDP session.  The "S"
    >      >>>>          attribute can be used by the application to relate all the
    >      >>>>          superimposed media streams enabling them to be added as an
    >      >>>> overlay on
    >      >>>>          top of any media stream.  The superimposition grouping
    >      >>>> semantics is
    >      >>>>          required, if the media data is separate and transported via
    >      >>>> different
    >      >>>>          sessions.
    >      >>>>
    >      >>>>
    >      >>>>
    >      >>>>
    >      >>>>       Please note that it may take a couple of minutes from the time
    >      >>>> of submission
    >      >>>>       until the htmlized version and diff are available at
    >      >>>> tools.ietf.org.
    >      >>>>
    >      >>>>       The IETF Secretariat
    >      >>>>
    >      >>>>
    >      >>>> _______________________________________________
    >      >>>> mmusic mailing list
    >      >>>> mailto:[email-masked]
    >      >>>> https://www.ietf.org/mailman/listinfo/mmusic
    >      >>>> _______________________________________________
    >      >>>> mmusic mailing list
    >      >>>> mailto:[email-masked]
    >      >>>> https://www.ietf.org/mailman/listinfo/mmusic
    >      >>>>
    >      >>>
    >      >>> _______________________________________________
    >      >>> mmusic mailing list
    >      >>> mailto:[email-masked]
    >      >>> https://www.ietf.org/mailman/listinfo/mmusic
    >      >>
    >      >> _______________________________________________
    >      >> mmusic mailing list
    >      >> mailto:[email-masked]
    >      >> https://www.ietf.org/mailman/listinfo/mmusic
    >      >
    >      > _______________________________________________
    >      > mmusic mailing list
    >      > mailto:[email-masked]
    >      > https://www.ietf.org/mailman/listinfo/mmusic
    >      > _______________________________________________
    >      > mmusic mailing list
    >      > mailto:[email-masked]
    >      > https://www.ietf.org/mailman/listinfo/mmusic
    >      >
    > 
    >      _______________________________________________
    >      mmusic mailing list
    >      mailto:[email-masked]
    >      https://www.ietf.org/mailman/listinfo/mmusic
    > 
    > _______________________________________________
    > mmusic mailing list
    > mailto:[email-masked]
    > https://www.ietf.org/mailman/listinfo/mmusic
    >