Subject: Re: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

From: Stephan Wenger <[email-masked]>
Date: Wed, 25 Aug 2021 21:32:31 +0000

Hi Miska,
Thanks for your excellent and detailed comments; please see inline in blue.  Yago/Ye-Kui: numerous action items for you inline as well.
Stephan

From: avt <[email-masked]> on behalf of Miska Hannuksela <[email-masked]>
Date: Tuesday, August 24, 2021 at 07:41
To: IETF AVTCore WG <[email-masked]>
Subject: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

Dear VVC RTP payload I-D authors and AVT experts,

I am responding to the WGLC as a contributor to the VVC standard and with the knowledge gained by authorship in AVC and HEVC RTP payload formats. Please find my technical comments below.

A few comments below are asking questions on particular design choices. Apologies if these have been already discussed in the past – I have not followed the development of the VVC RTP payload draft closely. In many cases, this might just be a sign to consider adding an informative note on why a design choice was made.

I have also sent a number of editorial fixes and suggestions directly to the authors.

1) Section 4.2

   A lower value of TID indicates a higher importance.

   More-important NAL units MAY be better protected against transmission

   losses than less-important NAL units.

      For Discussion: quite possibly something similar can be said for

      the Layer_id in layered coding, but perhaps not in multiview

      coding.  (The relevant part of the spec is relatively new,

      therefore the soft language).  However, for serious layer pruning,

      interpretation of the VPS is required.  We can add language about

      the need for stateful interpretation of LayerID vis-a-vis

      stateless interpretation of TID later.

  *   Note that independent layers are also possible, e.g. for depth maps or alpha planes.

  *   In VVC, VPS does not indicate the type of scalability. Consequently, layer pruning in a MANE needs to be based on VPS and some additional signaling, such as the SEI messages currently under development in JVET. Since these SEI messages are work in progress, it is proposed simply to remove the paragraph for discussion and not discuss layer pruning or layer importance in the RTP payload text.

I think your proposed solution is a good one.  I think we will implement unless we hear differently.

2) Section 7.1, profile-id semantics

         A profile_tier_level( ) syntax structure may be contained in an

         SPS, VPS, or DCI NAL units as specified in [VVC].  One of the

         following three cases applies to the container NAL unit of the

         profile_tier_level( ) syntax structure containing those PTL

         syntax elements used to derive the values of profile-id, tier-

         flag, level-id, sub-profile-id, or interop-constraints:

         ...

         3) The container NAL unit is a DCI NAL unit and the

         profile_tier_level( ) syntax structures in all DCI NAL units in

         the bitstream has the same values respectively for those PTL

         syntax elements.

  *   The first sentence above might give a wrong impression, since VPS and DCI may contain multiple PTL syntax structures. The sentence could be rephrased as follows: “As specified in [VVC], a profile_tier_level( ) syntax structure may be contained in an SPS NAL unit, and one or more profile_tier_level( ) syntax structures may be contained in a VPS NAL unit and in a DCI NAL unit.”

It's of course correct that any of the parameter sets mentioned can include more than one PTL structure, and we will adjust the text accordingly.
I don’t recall why multiple PTLs were allowed, but I don’t think it’s critical for an RTP-based system, where real-time encoders are the norm and splicing the exception.  Perhaps the easiest way to deal with problems arising from potentially multiple PTL structures (as you outline below), is to discourage those bitstreams.  Something like “VVC bitstreams transported over RTP using the technologies of this memo SHOULD contain only a single PTL structure in the DCI, VPS, and SPS.  The VVC syntax is more flexible, in allowing multiple PTL structures, but some of the mechanisms described herein would show undefined behavior.  If a system were to employ multiple PTL structures contrary to above “SHOULD”, the encoder(s) need to ensure that the mechanisms of this payload format, which are designed for only a single PTL structure, continue to work, for example by populating the multiple PTL structures with non-contradictory information.”

  *   I am a bit uncertain how to interpret the phrasing “the PTL syntax structures in all DCI NAL units in the bitstream has the same values respectively for those PTL syntax structures”.
     *   Let’s take an example: there are two DCI NAL units, dciA and dciB, in the bitstream, and both of them contain two PTL structures, i.e. dciA contains ptlA1 and ptlA2 structures (in this order) and dciB contains ptlB1 and ptlB2 structures (in this order).
     *   Does the phrasing mandate ptlA1 to be the same as ptlB1, and ptlA2 to be the same as ptlB2? If so, then why is the phrasing needed at all, since the VVC standard requires all DCI NAL units in a bitstream to have the same content.
     *   Or does the phrasing mandate ptlA1 to be the same as ptlA2, and ptlB1 to be the same as ptlB2? This interpretation would not be meaningful technically, since why would an encoder include multiple PTL structures in a DCI NAL unit, if they the same values.

I think this would be solved through the above limitation.

  *   It seems that case 3 (DCI) would not work if a DCI NAL unit contains more than one PTL syntax structure. If there were multiple PTL structures in a DCI NAL unit, how would it be determined which one of them is used to derive profile-id, tier- flag, level-id, sub-profile-id, and interop-constraints? Would case 3 need to be limited to apply only if DCI NAL unit(s) contain only one PTL structure?

Yes, to the second question, and again, the problem would be solved through above limitation.

3) Section 7.1, tier-flag, level-id

         If the tier-flag and level-id are derived from the

         profile_tier_level( ) syntax structure in a DCI NAL unit, the

         following applies:

  *   According to VVC, a DCI NAL unit may contain multiple PTL syntax structures. This sentence seems to assume that there would be only one PTL structure in a DCI NAL unit.

Solved per point 2, above.

4) Section 7.1, sprop-ols-id

         The value of sprop-ols-id MUST be in the range of 0 to 257,

         inclusive.

            Informative note: VVC allows having up to 258 output layer

            sets indicated in the VPS as the number of output layer sets

            minus 2 is indicated with a field of 8 bits.

  *   The maximum value of vps_num_output_layer_sets_minus2 is 255. Thus, VVC allows to have 257 output layer sets and the value range for OLS ID is from 0 to 256, inclusive.

That numbering game again :-(. You may well be right.  Ye-Kui/Yago?

5) Section 7.1, recv-ols-id

         When present, the value of recv-

         ols-id must be included only when sprop-ols-id was received and

         must refer to an output layer set in the VPS that is in the

         same dependency tree as the OLS referred to by sprop-ols-id.

  *   I don’t understand why sprop-ols-id is required in the offer in order to have recv-ols-id in the answer. Could you explain?
     *   Note that if this requirement is relaxed, Section 7.2.2.2 would also need changes.

  *   In my understanding:
     *   The absence of sprop-ols-id in an offer indicates that the offerer can transmit any OLS that is included in sprop-vps. Consequently, the answerer could respond with recv-ols-id equal to any OLS ID included in sprop-vps.
     *   The presence of sprop-ols-id in an offer indicates that the offerer only has a bitstream corresponding to the OLS indicated by sprop-ols-id and can transmit only that OLS or any OLS contains either the same layers as or a subset of the layers of the OLS indicated by sprop-ols-id.

Yago, can you take this?  I vaguely recall discussions around this, but not the conclusion.

  *   The term dependency tree is not defined and hence should either be avoided in the phrasing or defined.

Does this work: “[…]must refer to an OLS in the VPS that includes no layers other than all or a subset of the layers of the OLS referred to by sprop-ols-id. “

6) Section 7.1, sprop-vps

         The sprop-vps parameter MAY contain one or more than one video

         parameter set NAL unit.  However, all other video parameter

         sets contained in the sprop-vps parameter MUST be consistent

         with the first video parameter set in the sprop-vps parameter.

         A video parameter set vpsB is said to be consistent with

         another video parameter set vpsA if any decoder that conforms

         to the profile, tier, level, and constraints indicated by the

         data starting from the syntax element general_profile_space to

         the syntax element general_level_idc, inclusive, in the first

         profile_tier_level( ) syntax structure in vpsA can decode any

         bitstream that conforms to the profile, tier, level, and

         constraints indicated by the data starting from the syntax

         element general_profile_space to the syntax element

         general_level_idc, inclusive, in the first profile_tier_level(

         ) syntax structure in vpsB.

  *   I don’t understand what makes the first PTL structure in a VPS special. Could you explain? Perhaps adding an informative NOTE in the text would help readers to understand this design choice.

Point 2 above should take care of this

  *   Shouldn't the consistency requirement to be stricter so that the number of OLSs in different VPSs must match and a decoder capable of decoding the PTL in vpsA for any OLS with index olsIdx can decode any CVSs referring to vpsB when TargetOlsIdx is equal to olsIdx?

Yes, I believe so,  Excellent catch.  Will fix.

  *   Shouldn’t “can decode any bitstream” rather be “can decode any CVS(s) referencing vpsB”? Referencing a new VPS does not necessarily start a new bitstream.

Correct.  Will fix.

7) Section 7.1, sprop-sei

            ... the display orientation SEI message

            when the device is a handheld device (as the display

            orientation may change when the handheld device is turned

            around), ...

  *   Neither the VVC standard nor the VSEI standard currently includes the display orientation SEI message. It is proposed to remove this part of the informative note.

Right.  Thanks. Will fix.

8) Section 7.1, related to sprop-sei

  *   It would be helpful to enable offer-answer negotiation on properties that are signaled with SEI messages. These properties could include e.g.:
     *   Whether film grain synthesis is supported (the film grain characteristics SEI message)
     *   Whether frame packing is supported and which frame packing arrangement type to use (the frame packing arrangement SEI message)
     *   How many subpictures to use and which levels they should correspond (subpicture level information SEI message)

  *   It is suggested to allow empty or truncated SEI message payloads in sprop-sei to indicate the capability of the offerer to encode these SEI messages with any SEI message payload that starts with the bits included in sprop-sei (if any). For example, when contained in sprop-sei:
     *   The film grain characteristics SEI message with a truncated payload that only contains data up to fg_model_id indicates that the offerer is capable of producing a bitstream containing film grain of that fg_model_id value.
     *   The frame packing arrangement SEI message with an empty payload indicates that the offerer is capable of producing a bitstream with any frame packing arrangement.
     *   The subpicture level information SEI message with an empty payload indicates that the offerer is capable of encoding content with subpictures and include subpicture level information SEI message(s) into the bitstream.

  *   It is suggested to introduce an optional recv-sei parameter along the following principles:
     *   The SEI messages included in recv-sei in an answer indicate which SEI messages must be present in the bitstream from the offerer to the answerer
     *   When recv-sei is present in an answer, the SEI message types in recv-sei must be the same as or a subset of those included in the sprop-sei of the offer.
     *   When recv-sei is present in an answer, the SEI message payload for a particular SEI message type in recv-sei must start with the SEI message payload bits present (if any) in recv-sei for the same SEI message type.
     *   It is suggested to allow empty or truncated SEI message payloads in recv-sei to indicate that the answerer has the liberty to encode any values for remaining of the SEI message payload (as long as they conform to the specification where the SEI message is specified). Note that the conditions in the bullets above still apply.
     *   When an SEI message with a particular SEI message type is present in sprop-sei of an offer and is absent in recv-sei in an answer, the answerer indicates that it does not support the processing the SEI message included in sprop-sei.

I see the motivation, but implementing any of these above points would be major surgery at the current stage.  Correct me if I’m wrong, but all this could also be “bolted on” later, with no detrimental effect to interop, AFAICT.  I’m a bit reluctant.  Others?

9) Section 7.2.2.1

      1) When sprop-sps parameter is present, all sequence parameter

      sets contained in the sprop-sps parameter in the SDP answer and

      all sequence parameter sets sent in-band for either the offerer-

      to-answerer direction or the answerer-to-offerer direction MUST be

      consistent with the first sequence parameter set in the sprop-sps

      parameter of the offer (see the semantics of sprop-sps in

      Section 7.1 of this document on one sequence parameter set being

      consistent with another sequence parameter set).

  *   The semantics of sprop-sps do not specify when an SPS is consistent with another SPS. It presumably means roughly the following: “An SPS spsB is consistent with an SPS spsA, when a decoder capable of decoding the PTL in spsA can decode any CLVSs referring to spsB.” Please check and add definition of consistent SPS into the semantics of sprop-sps.

Your definition seems right to me, but could Yago & Ye-Kui please weight in also?  And yes, we can add this language.

10) Section 7.2.2.2, on sprop-sub-layer-id indicating the property of the highest layer only

   o  The offerer MAY include sprop-sub-layer-id which, in case of

      scalable VVC, is interpreted as the highest sub-layer of the

      highest enhancement layer in the OLS indicated by sprop-ols-id.

      The answerer MAY include recv-sub-layer-id which can be used to

      downgrade the sublayer of the highest enhancement layer.  This

      specification does not support downgrading the sub-layer of any

      layers in the OLS that are not the highest layer.

         Informative note: in other words, using this mechanism, an

         answerer can downgrade only the frame rate for the highest

         spatial/quality layer (typically corresponding to the highest

         resolution or bitrate, hence the most complex to decode), but

         not for lower spatial/quality layers.  The answerer must

         support all sublayers for lower layers in the OLS, or reject

         the offer.  That's not a big burden, as the receiver/decoder

         has the option to discard any sublayers it cannot decode,

         irrespective of what is being signalled through offer/answer.

  *   This design choice is not obvious. Note that pruning temporal sublayers from the highest layer but not from the reference layer(s) could have undesirable impacts, such as:
     *   The decoder outputs reference-layer pictures with TemporalId greater than sprop-sub-layer-id. In SNR/spatial scalability these pictures have lower quality/resolution than the pictures of the highest layer and hence cause temporally fluctuating quality/resolution.
     *   If sprop-sub-layer-id is used to input the Htid variable to the decoder as specified in Section 8.1.1 of VVC, the receiver would need to apply the sub-bitstream extraction process of VVC, C.6 to obtain a conforming bitstream.

  *   I would think that the maximum of sprop-sub-layer-id (when present) and recv-sub-layer-id (when present) MUST indicate the highest temporal sublayer to be decoded for the bitstream from the offerer to the answerer and MAY be provided as the value of Htid to the VVC decoding process as specified in Section 8.1.1 of VVC. Changes would be needed in Section 7.2.2.2 as well as in sprop-sub-layer-id and recv-sub-layer-id in Section 7.1.

Umm.  I’m lost.  Yago/Ye-Kui?

11) Section 7.2.3

      *  Declaring actual configuration or bitstream properties:

         +  profile-id

         +  tier-flag

         +  level-id

         +  interop-constraints

         +  sub-profile-id

         +  sprop-dci

         +  sprop-vps

         +  sprop-sps

         +  sprop-pps

         +  sprop-max-don-diff

         +  sprop-depack-buf-bytes

         +  sprop-sub-layer-id

         +  sprop-ols-id

  *   Should the list also contain sprop-sei?

Yes.  Will fix.

12) Section 10

   A media sender can remove NAL units

   belonging to higher temporal sublayers (i.e., those NAL units with a

   high value of TID) or higher spatio-SNR layers (as indicated by

   interpreting the VPS) until the sending bitrate drops to an

   acceptable range.

  *   VPS does not indicate the type of scalability. It is proposed to remove “(as indicated by interpreting the VPS)”.

We can do that.

Best Regards,
Miska Hannuksela

From: avt <[email-masked]> On Behalf Of Bernard Aboba
Sent: Monday, August 16, 2021 11:30 PM
To: IETF AVTCore WG <[email-masked]>
Subject: [AVTCORE] WGLC on “RTP Payload Format for Versatile Video Coding (VVC)”

This is an announcement of WG last call on "RTP Payload Format for Versatile Video Coding (VVC)”.

The document is available for inspection here:

https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-vvc/

WG Last Call will end on August 30, 2021.

In response, please state one of the following:

* I support advancing the document to Proposed Standard

* I object to advancement to Proposed Standard, due to Issues

described below <Issue description or link>.

Bernard Aboba

For the Chairs

---

From: "Ye-Kui Wang" <[email-masked]>
Date: Thu, 26 Aug 2021 16:03:38 -0700

Hi Miska,

Thanks for reviewing the draft! Here are my responses regarding your comments for which Stephan has called my name.

4) You are right. VVC allows to have 257 output layer sets and the value range for OLS ID is from 0 to 256, inclusive.

9) Good catch. The current text includes such a definition for VPS, but misses a similar definition for SPS. Should be added (similarly as Miska suggested and similarly as the definition for VPS).

10) I leave this item for Yago 😊

BR, YK

From: Stephan Wenger <[email-masked]> 
Sent: Wednesday, August 25, 2021 14:33
To: Hannuksela, Miska (Nokia - FI/Tampere) <[email-masked]>; IETF AVTCore WG <[email-masked]>
Cc: shuaiizhao(Shuai Zhao) <[email-masked]>; [email-masked]; [email-masked]
Subject: [External] Re: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

Hi Miska,

Stephan

From: avt <[email-masked] <mailto:[email-masked]> > on behalf of Miska Hannuksela <[email-masked] <mailto:[email-masked]> >
Date: Tuesday, August 24, 2021 at 07:41
To: IETF AVTCore WG <[email-masked] <mailto:[email-masked]> >
Subject: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

1) Section 4.2

         ...

         syntax elements.

 

         following applies:

 

         inclusive.

*	In my understanding:

 

Correct.  Will fix.

            around), ...

Right.  Thanks. Will fix.

9) Section 7.2.2.1

11) Section 7.2.3

         +  profile-id

         +  tier-flag

         +  level-id

         +  interop-constraints

         +  sub-profile-id

         +  sprop-dci

         +  sprop-vps

         +  sprop-sps

         +  sprop-pps

         +  sprop-max-don-diff

         +  sprop-depack-buf-bytes

         +  sprop-sub-layer-id

         +  sprop-ols-id

Yes.  Will fix.

12) Section 10

   acceptable range.

We can do that.

Best Regards,

Miska Hannuksela

From: avt <[email-masked] <mailto:[email-masked]> > On Behalf Of Bernard Aboba
Sent: Monday, August 16, 2021 11:30 PM
To: IETF AVTCore WG <[email-masked] <mailto:[email-masked]> >
Subject: [AVTCORE] WGLC on “RTP Payload Format for Versatile Video Coding (VVC)”

 

 
Bernard Aboba
 
For the Chairs

---

From: "Sanchez de la Fuente, Yago" <[email-masked]>
Date: Fri, 27 Aug 2021 14:06:00 +0000

Hi Miska, all,

Thanks Miska for reviewing the draft. Please see my answers inline.

Best regards,
Yago Sánchez

---
Department Video Communication and Applications
Group Multimedia Communications

Fraunhofer HHI - Heinrich Hertz Institute
Einsteinufer 37, 10587 Berlin, Germany
http://www.hhi.fraunhofer.de/ip/mc

Tel.: +49 30 310 02663
[email-masked]<mailto:[email-masked]>

On 27. Aug 2021, at 01:03, Ye-Kui Wang <[email-masked]<mailto:[email-masked]>> wrote:

Hi Miska,

BR, YK

From: Stephan Wenger <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, August 25, 2021 14:33
To: Hannuksela, Miska (Nokia - FI/Tampere) <[email-masked]<mailto:[email-masked]>>; IETF AVTCore WG <[email-masked]<mailto:[email-masked]>>
Cc: shuaiizhao(Shuai Zhao) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [External] Re: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

Hi Miska,

From: avt <[email-masked]<mailto:[email-masked]>> on behalf of Miska Hannuksela <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, August 24, 2021 at 07:41
To: IETF AVTCore WG <[email-masked]<mailto:[email-masked]>>
Subject: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

1) Section 4.2

[YS] I think that Stephan’s suggestion here is good while I think that it only is needed for DCI. We might have several PTLs in VPS for different OLSs and that should be fine.

[YS] I think this is ok, but I am unsure what would we say when there is more than on PTL. What would we exactly mean with “for example by populating the multiple PTL structures with non-contradictory information”? Would it  mean that all PTLs are the same? In that case we are good. But if the meaning of non-contradictory PTLs is more relaxed, we might need to explain here a bit more. Let’s for instance take as an example two PTLs in a DCI with each indicating a different sub-profile. I would understand non-contradictory information that one of these profiles needs to be a subset of the other and not simply “disjoint” profiles. Then we would need to say that the sub-profile in the SDP needs to be the one that includes the other sub-profile. Unless we completely disallow different PTLs in a DCI, in which case the current text is ok since both sub-profiles would be exactly the same. If the later, we should clearly indicate that two PTLs in DCI are not supported but the “SHOULD” above would not be enough I think. Which one should we do?

[YS] I also confirm that this needs to be fixed. Good catch!

  *   In my understanding:

[YS] We currently have a default value for sprop-ols-id in section 7.1, which follows the same inference rule as TargetOlsIdx in 8.1.1 in [VVC], so this means to me that the absence does not necessarily indicate that any OLS included in the sprop-vps can be transmitted. At least this was not my recollection. If we wanted to do so we should remove the inference value and state this clearly.  As for why we took this design choice, I remember we had some discussion about OLS negotiation and our intention was to keep things simple. We wanted to simplify the OLS negotiation and we decided that 1) the presence of sprop-ols-id was indicating the willingness of the sender to negotiate OLSs and the absence of it meant that that flexibility was not desired and “regular” negotiation should be carried out where the media format is included as is in the answer or removed completely (when one or more of the parameter values are not supported); and 2) when negotiating OLSs not anyone could be selected by the answerer but only a subset of the offered one (the text regarding the dependency tree).

[YS] This is a carry over from the HEVC payload format which only applied to the single layer-case and therefore the first PTL structure was mentioned there. We have more than on PTL in layered bitstream for VPSs but probably we need to do as suggested below and be stricter on all PTLs in a VPS.

Correct.  Will fix.

Right.  Thanks. Will fix.

[YS] I agree with Stephan. I understand the motivation but this is quite a big change. In particular, I was wondering whether allowing empty or truncated SEI messages is necessary or whether the SEI manifest SEI message or SEI prefix indication SEI message could be used for this purpose.

9) Section 7.2.2.1

[YS] We had some internal discussion before the last update and the point that was made is that it seems that some implementations using scalability use pruning of the higher layers mainly. Having said that I agree with the issues pointed out.

So
1) Yes you would have temporally fluctuating quality/resolution
2)  sprop-sub-layer-id cannot be used to set Htid and the extraction process drops some sublayers of the highest layer, which is not how it is defined in VVC. Basically, sprop-sub-layer-id would not change the operating point and picture from non-output layers need to be output.

So if we keep this design choice we need to clarify this; that we are doing something different than how VVC defines operating points. My recollection is that this is how some products are implemented and differs from how VVC defines operating points. Stephan could you confirm that my recollection is correct?

So not sure how to proceed here. If this is how it is typically implemented, it seems reasonable to do things a bit different than how operating points are defined in VVC, but we should clearly state it. Any views on how to proceed here?

11) Section 7.2.3

Yes.  Will fix.

12) Section 10

We can do that.

Best Regards,
Miska Hannuksela

From: avt <[email-masked]<mailto:[email-masked]>> On Behalf Of Bernard Aboba
Sent: Monday, August 16, 2021 11:30 PM
To: IETF AVTCore WG <[email-masked]<mailto:[email-masked]>>
Subject: [AVTCORE] WGLC on “RTP Payload Format for Versatile Video Coding (VVC)”

Bernard Aboba

For the Chairs

---

From: "Hannuksela, Miska (Nokia - FI/Tampere)" <[email-masked]>
Date: Mon, 30 Aug 2021 18:37:46 +0000

Hi Stephan, Ye-Kui, Yago,

Thanks for your responses. Below I am continuing the discussion on items that have not been concluded yet.

[MH] I agree with Yago that discouraging the use of multiple PTLs only concerns DCI NAL units.

[MH] Note that this part of the I-D discusses how the sender can derive the value for profile-id from 1) SPS, 2) VPS, or 3) DCI. Using case 1 or 2 for deriving profile-id is always possible, while DCI with a single PTL can simplify the operation. I’d tend to think that allowing multiple PTLs in DCI NAL unit is still allowed but discouraged (as Stephan suggests above). If a DCI NAL unit contains multiple PTLs, the sender needs to derive the profile-id from SPS or VPS.

  *   In my understanding:

[MH] Thanks for the explanation. With Stephan’s definition of dependency tree (below), I am able to understand all the pieces now. I don’t have strong opinions on this issue. If design stability is preferred, the current text can be kept except that I find the inference of sprop-ols-id confusing, since the inferred value is not used for anything as far as I can see. Therefore I would propose to delete the sentence on inferring the sprop-ols-id value. Or am I missing something?

[MH] Sounds good, thanks.

[MH] Right, Yago, the already concluded point 9 should take care of this without treating the first PTL in a VPS in a special way.

[MH] Good point, Yago, on the SEI manifest and SEI prefix indication SEI messages. I think they can indeed be used to avoid the empty or truncated SEI messages in sprop-sei and recv-sei and simplify the design. I’d think no change in sprop-sei is needed and recv-sei is simplified to the following:

     *   The SEI messages included in recv-sei in an answer indicate which SEI messages must be present in the bitstream from the offerer to the answerer
     *   When recv-sei is present in an answer, the SEI message in recv-sei must be one of the following:

        *   the SEI message must be the same as directly included in the sprop-sei of the offer;
        *   the SEI message must have a type indicated in a SEI manifest SEI message in the sprop-sei of the offer;
        *   the SEI message must have a type and start with the respective content indicated by a SEI prefix indication SEI message contained within the srop-sei.

I continue to think this would be useful for many purposes. I am opening up two of the cases I briefly mentioned above as examples:

  *   The offerer encodes stereoscopic video, with spatial or temporal frame packing. It includes SEI manifest SEI message with frame packing arrangement SEI message in sprop-sei. The answerer can deal with spatial frame packing, but temporal frame packing is not supported by its displaying process. It includes the frame packing arrangement SEI message with a spatial packing type in recv-sei of the answer.
  *   The offerer is able to encode 8K video and supports encoding of independent subpictures. It includes SEI manifest SEI message with subpicture level information SEI message in sprop-sei. The answerer can decoded 8K video only if it uses four 4K-capable processing cores, one processing core per each subpicture sequence. Thus, it includes subpicture level information SEI message indicating four subpictures, each with a level of 4K decoding capacity in the recv-sei of the answer.

So

[MH] I am having hard time to understand the motivation why only the highest layer would undergo temporal sublayer pruning. And if so, would the temporally fluctuating quality/resolution be really a desirable way to display such video. Unless there are good answers, I think the I-D should be aligned with the way VVC defines operating points.

Best Regards,
Miska

---

From: "Ye-Kui Wang" <[email-masked]>
Date: Tue, 31 Aug 2021 11:12:57 -0700

Hi Miska, Yago, Stephan,

On item 8, indeed what Miska you suggested is aligned with the purpose of the SEI manifest SEI message and the SEI prefix indication SEI message. These two SEI messages were initially introduced into HEVC, and later added to AVC, and then to VVC version 2 (which is expected to be finalized in Jan. 2022). 

Given that 1) an SEI message based offer/answer mechanism would be useful for almost all video coding standards, particularly AVC, HEVC, VVC and their extensions, and 2) in this mechanism the SEI manifest SEI message and the SEI prefix indication SEI message should be used for AVC, HEVC, VVC and their extensions, while VVC v2 is not yet finalized but will be soon, I think it’s probably better to leave this aspect outside the VVC RTP payload format, but work towards a separate I-D/RFC that would work for all video codecs, and for AVC, HEVC, VVC and their extensions, the design is based on the SEI manifest SEI message and the SEI prefix indication SEI message.

BR, YK

From: Hannuksela, Miska (Nokia - FI/Tampere) <[email-masked]> 
Sent: Monday, August 30, 2021 11:38
To: Sanchez de la Fuente, Yago <[email-masked]>; Ye-Kui Wang <[email-masked]>; Stephan Wenger <[email-masked]>
Cc: IETF AVTCore WG <[email-masked]>; shuaiizhao(Shuai Zhao) <[email-masked]>
Subject: RE: [External] Re: [AVTCORE] Comments on “RTP Payload Format for Versatile Video Coding (VVC)”

         ...

         syntax elements.

 

 

 

*	In my understanding: 

 

 

 

So 

 

 

Best Regards,

Miska