Subject: Re: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt

From: Tom Kristensen <[email-masked]>
Date: Tue, 14 Sep 2010 15:01:06 +0200

On 14 September 2010 14:45,  <[email-masked]> wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>        Title           : Additional H.241 Parameter in the RTP Payload Format for H.264 Video
>        Author(s)       : T. Kristensen, M. Walters
>        Filename        : draft-kristensen-avt-rtp-h241param-00.txt
>        Pages           : 9
>        Date            : 2010-09-14
>
> This document defines a new optional parameter addressing recent
> extensions currently supported in H.323 systems: The signalling of
> the maximum frames per second that can be efficiently received or
> that can be sent.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-kristensen-avt-rtp-h241param-00.txt

This draft simply specify how to use and interpret the parameter
MaxFPS, specified in the latest amendment to H.241, in SDP-based
systems. Such a parameter to express maximum frame rate is not present
in the - soon to enter RFC state - 3984bis draft.

Current parameters/attributes for maximum frame rate signalling are
described, shortcomings analysed and the motivation for this new
optional parameter in the H.264 RTP payload format are presented.

Your feedback and comments are more than welcomed!

Cheers,
-- Tom

-- 
# TANDBERG, now part of Cisco
## [email-masked]  |  http://www.tandberg.com
### http://folk.uio.no/tomkri/

---

From: Ye-Kui Wang <[email-masked]>
Date: Wed, 15 Sep 2010 05:32:26 -0400

Hi Tom, All,

I have a comment. The essentail reason I found from the draft for the
addition of the parameter of max frame rate is as follows: 

   Increasingly, systems are able to handle 60 frames per
   second and there is no feasible way to discover whether a system can
   handle such frame rates or not to ensure interoperability and to
   optimise resolution choices.

   In many cases there is a maximum frame rate at which the decoder side
   can use data.  **This limit** may stem from internal architecture or
   external factors, e.g. type of display attached.  If the encoder
   chooses to send at a higher frame rate than this maximum limit, the
   decoder side will normally discard the additional frames after
   decoding them.  The transmission of the extra frames and the
   processing of frames to just discard them is wasteful and the
   bandwidth and processing could be used more effectively.

**The limit** (on the max frame rate the system can effectively handly) that
stems from internal architecture (of the systems) or external factors (e.g.
type of display attached) is not because of conforming video decoder
implementations. For a system that has a limit on max frame rate, using the
generic SDP parameter a=framerate is good enough. Therefore, I don't see a
good reason to define a new parameter on max frame rate specific to
H.264/AVC.

BR, YK 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Tom Kristensen
> Sent: Tuesday, September 14, 2010 9:01 AM
> To: IETF AVT WG
> Cc: Tom Kristensen; Malcolm Walters
> Subject: Re: [AVT] I-D 
> Action:draft-kristensen-avt-rtp-h241param-00.txt
> 
> >        Title           : Additional H.241 Parameter in the 
> RTP Payload 
> > Format for H.264 Video
> > extensions currently supported in H.323 systems: The 
> signalling of the 
> > maximum frames per second that can be efficiently received 
> or that can 
> > be sent.
> >
> This draft simply specify how to use and interpret the 
> parameter MaxFPS, specified in the latest amendment to H.241, 
> in SDP-based systems. Such a parameter to express maximum 
> frame rate is not present in the - soon to enter RFC state - 
> 3984bis draft.
> 
> Current parameters/attributes for maximum frame rate 
> signalling are described, shortcomings analysed and the 
> motivation for this new optional parameter in the H.264 RTP 
> payload format are presented.
> 
> ## [email-masked]  |  http://www.tandberg.com ### 
> http://folk.uio.no/tomkri/ 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>

---

From: Tom Kristensen <[email-masked]>
Date: Wed, 15 Sep 2010 15:08:55 +0200

YK,

Just a quick counter question, before answering your concerns below:
Do you see no potential problems or confusion when using the a=framerate 
attribute in cases where different codecs have different max frame rate 
capabilities?

-- Tom

On 09/15/2010 11:32 AM, Ye-Kui Wang wrote:
>> On 14 September 2010 14:45,<[email-masked]>  wrote:

---

From: Ye-Kui Wang <[email-masked]>
Date: Wed, 15 Sep 2010 10:30:19 -0400

Tom,

I don't see a problem or confusion when using a=framerate attribute in cases
where different payload types have different max frame rates, as in that
case the value of a=framerate can (and should) be equal to the max of the
max frame rates.

BR, YK 

> From: Tom Kristensen [mailto:[email-masked]] 
> Sent: Wednesday, September 15, 2010 9:09 AM
> To: Ye-Kui Wang
> Cc: 'Tom Kristensen'; 'IETF AVT WG'; 'Malcolm Walters'
> Subject: Re: [AVT] I-D 
> Do you see no potential problems or confusion when using the 
> a=framerate attribute in cases where different codecs have 
> different max frame rate capabilities?
> 
> -- Tom
> 
> > **The limit** (on the max frame rate the system can effectively 
> > handly) that stems from internal architecture (of the 
> systems) or external factors (e.g.
> > implementations. For a system that has a limit on max frame rate, 
> > using the generic SDP parameter a=framerate is good enough. 
> Therefore, 
> > I don't see a good reason to define a new parameter on max 
> frame rate 
> > specific to H.264/AVC.
> >
> > BR, YK
> >
> >> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of 
> >> Tom Kristensen
> >>>         Filename        : 
> draft-kristensen-avt-rtp-h241param-00.txt
> >>>         Pages           : 9
> >>>         Date            : 2010-09-14
> >>>
> http://www.ietf.org/internet-drafts/draft-kristensen-avt-rtp-h241para
> >> m
> >>> -00.txt
> >>
> >> systems. Such a parameter to express maximum frame rate is not 
> >> present in the - soon to enter RFC state - 3984bis draft.
> >>
> signalling are

---

From: Tom Kristensen <[email-masked]>
Date: Thu, 16 Sep 2010 13:54:41 +0200

[...]
> good reason to define a new parameter on max frame rate specific to

You have a point here with regards to the external factors. If the 
limiting factor is the display, of course the max framerate for all 
codecs will be the same. (However, one of the codec implementations may 
have a different and lower limit than the rest).

The restriction from the internal architecture may not necessarily come 
from the overall system architecture. It might be the codec as such, 
i.e. the design and/or implementation of the H.264 decoder in this case.

-- Tom

---

From: Tom Kristensen <[email-masked]>
Date: Thu, 16 Sep 2010 14:19:43 +0200

> case the value of a=framerate can (and should) be equal to the max of the

Yes, this interpretation is valid. The text added in Section 5 of the 
draft 
(http://tools.ietf.org/html/draft-kristensen-avt-rtp-h241param-00#section-5) 
will then be changed to something along the lines of:

   'The "a=framerate" limitation allows a global maximum
   frame rate to be specified. This limit applies to all
   codecs. In cases where both "a=framerate" and codec specific
   limits are specified, the maximum frame rate that can be used
   for a codec is the minimum of the values specified globally
   and specifically for a codec.'

Then imagine an upcoming H.265, VC-2, VC-666 or whatever video standard 
becomes available and typical frame rates rise to above 60 fps. In such 
cases we are stuck with no way of signalling a H.264 specific max frame 
rate, as in an SDP video media section "a=framerate" will then be 120 
fps or something and your existing H.264 implementation might just cope 
with 60 fps or even below.

As shown in the text H.261 and H.263 have a way to signal a maximum 
frame rate - explicit or implicit - today. We propose the same mechanism 
for H.264.

Another interesting fact is the following note added in RFC3984 (and 
-bis) for the max-br parameter:

           'Informative note: This parameter was added primarily
           to complement a similar codepoint in the ITU-T
           Recommendation H.245, so as to facilitate signaling
           gateway designs. [..]'

As max-fps is used today in H.323 systems today, there is a similar 
motivation for adding max-fps with the same semantics to ease the 
gatewaying of SIP and H.323.

-- Tom

---

From: Stephen Botzko <[email-masked]>
Date: Thu, 16 Sep 2010 09:59:41 -0400

I think that is the problem.  If you can decode H.263 at 60 fps, but are
limited to H.264 at 30 fps, then a=60 is what you need to signal.

To limit H.264 to 30, you'd need to downgrade your profile/level, which has
negative consequences for compression efficiency.  This contribution
eliminates the need to do that.

Regards,
Stephen Botzko

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 09:13:57 -0400

> The restriction from the internal architecture may not 
> necessarily come from the overall system architecture. It 
> might be the codec as such, i.e. the design and/or 
> implementation of the H.264 decoder in this case.

Any H.264 decoder complaint to a certain profile and level must be able to
handle all bitstreams conforming to that profile and level. That is, a hard
limit on frame rate (regardless of spatial resoltuion) does not apply to
**conforming** H.264 decoders. 

BR, YK

> Sent: Thursday, September 16, 2010 7:55 AM
> To: Ye-Kui Wang
> You have a point here with regards to the external factors. 
> If the limiting factor is the display, of course the max 
> framerate for all codecs will be the same. (However, one of 
> the codec implementations may have a different and lower 
> limit than the rest).
> 
> The restriction from the internal architecture may not 
> necessarily come from the overall system architecture. It 
> might be the codec as such, i.e. the design and/or 
> implementation of the H.264 decoder in this case.
> 
> -- Tom
>

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 10:06:54 -0400

Tom, 

Good argument about H.245/H.323. However, as I mentioned in my previous
email (sent out a mnute ago), any H.264 decoder complaint to a certain
profile and level must be able to handle all bitstreams conforming to that
profile and level. That is, a hard limit on frame rate (regardless of
spatial resoltuion) does not apply to **conforming** H.264 decoders. Thus
still I don't see a reason to define the new parameter specifically for
H.264. 

BR, YK

> Sent: Thursday, September 16, 2010 8:20 AM
> To: Ye-Kui Wang
> > I don't see a problem or confusion when using a=framerate 
> attribute in 
> > cases where different payload types have different max 
> frame rates, as 
> > in that case the value of a=framerate can (and should) be 
> equal to the 
> > max of the max frame rates.
> 
> Yes, this interpretation is valid. The text added in Section 
> 5 of the draft
> Then imagine an upcoming H.265, VC-2, VC-666 or whatever 
> video standard becomes available and typical frame rates rise 
> to above 60 fps. In such cases we are stuck with no way of 
> signalling a H.264 specific max frame rate, as in an SDP 
> video media section "a=framerate" will then be 120 fps or 
> something and your existing H.264 implementation might just 
> cope with 60 fps or even below.
> 
> As shown in the text H.261 and H.263 have a way to signal a 
> maximum frame rate - explicit or implicit - today. We propose 
> the same mechanism for H.264.
> 
> As max-fps is used today in H.323 systems today, there is a 
> similar motivation for adding max-fps with the same semantics 
> to ease the gatewaying of SIP and H.323.
> 
> -- Tom
>

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 10:12:18 -0400

>> To limit H.264 to 30, you'd need to downgrade your profile/level, which
has negative consequences for compression efficiency.  This contribution
eliminates the need to do that.

Could you please give an example system that can decode H.263 at 60 fps but
is limited to H.264 at 30 fps regardless of the spatial resolution?
 
BR, YK

  _____  

From: Stephen Botzko [mailto:[email-masked]] 
Sent: Thursday, September 16, 2010 10:00 AM
To: Ye-Kui Wang
Cc: Tom Kristensen; IETF AVT WG; Malcolm Walters
Subject: Re: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt

Regards,
Stephen Botzko

Tom,

BR, YK

---

From: "Bill Ver Steeg (versteb)" <[email-masked]>
Date: Mon, 20 Sep 2010 09:14:01 -0500

There may be situations in which a nominally compliant decoder is only
able to handle a portion of the maximum frame rate. For instance, it may
be compositing multiple images. It may be sharing the CPU with other
applications.

Bvs

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ye-Kui Wang
Sent: Monday, September 20, 2010 10:07 AM
To: 'Tom Kristensen'
Cc: 'IETF AVT WG'; 'Malcolm Walters'
Subject: Re: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt

Tom, 

Good argument about H.245/H.323. However, as I mentioned in my previous
email (sent out a mnute ago), any H.264 decoder complaint to a certain
profile and level must be able to handle all bitstreams conforming to
that
profile and level. That is, a hard limit on frame rate (regardless of
spatial resoltuion) does not apply to **conforming** H.264 decoders.
Thus
still I don't see a reason to define the new parameter specifically for
H.264. 

BR, YK

---

From: Roni Even <[email-masked]>
Date: Mon, 20 Sep 2010 17:00:41 +0200

Hi,
Table A.1 in H.264 defines the levels. There is a definition of maximum
MaxMBPS, MaxFS and MaxBR. There is not definition of what is the maximum
frame rate, I assume that this can be achieved by limiting the MaxMBPS and
MaxFS parameters that are defined in  H.241 and 3984bis. This is what table
A.6 in H.264 provides so why the extra parameter.

Roni Even
As individual

> Sent: Monday, September 20, 2010 4:07 PM
> To: 'Tom Kristensen'

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 11:08:04 -0400

> For instance, it may be compositing multiple images. 

What do you mean by a video decoder "composing multiple images"? 

> It may be sharing the CPU with other applications.

If a video decoder shares the CPU with other applications, the system would
not know how much computing resource would be available for decoding. In
that case, I am not sure how can the system even ensure that the indicated
max frame rate be properly handled. 

BR, YK 

> From: Bill Ver Steeg (versteb) [mailto:[email-masked]] 
> Sent: Monday, September 20, 2010 10:14 AM
> To: Ye-Kui Wang; Tom Kristensen
> Cc: IETF AVT WG; Malcolm Walters
> Subject: RE: [AVT] I-D 
> There may be situations in which a nominally compliant 
> decoder is only able to handle a portion of the maximum frame 
> rate. For instance, it may be compositing multiple images. It 
> may be sharing the CPU with other applications.
> 
> Bvs
> 
> 
> Behalf Of Ye-Kui Wang
> Good argument about H.245/H.323. However, as I mentioned in 
> my previous email (sent out a mnute ago), any H.264 decoder 
> complaint to a certain profile and level must be able to 
> handle all bitstreams conforming to that profile and level. 
> That is, a hard limit on frame rate (regardless of spatial 
> resoltuion) does not apply to **conforming** H.264 decoders.
> Thus
> still I don't see a reason to define the new parameter 
> specifically for H.264. 
> 
> BR, YK
> 
> > Then imagine an upcoming H.265, VC-2, VC-666 or whatever video 
> > standard becomes available and typical frame rates rise to above 60 
> > fps. In such cases we are stuck with no way of signalling a H.264 
> > specific max frame rate, as in an SDP video media section 
> > "a=framerate" will then be 120 fps or something and your existing 
> > H.264 implementation might just cope with 60 fps or even below.
> > 
> > frame rate - explicit or implicit - today. We propose the same 
> > mechanism for H.264.
> >

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 11:13:14 -0400

Right, this is my piont as well. A decoder must be able to handle all
possible combinations of spatial resolution and frame rate (under the
constraints by MaxMBPS and MaxFS) to claim itself as conforming to the
standard. 

BR, YK 

> From: Roni Even [mailto:[email-masked]] 
> Sent: Monday, September 20, 2010 11:01 AM
> To: 'Tom Kristensen'
> Cc: 'IETF AVT WG'; 'Malcolm Walters'; 'Ye-Kui Wang'
> Subject: RE: [AVT] I-D 
> Table A.1 in H.264 defines the levels. There is a definition 
> of maximum MaxMBPS, MaxFS and MaxBR. There is not definition 
> of what is the maximum frame rate, I assume that this can be 
> achieved by limiting the MaxMBPS and MaxFS parameters that 
> are defined in  H.241 and 3984bis. This is what table
> Behalf Of 
> > Good argument about H.245/H.323. However, as I mentioned in my 
> > previous email (sent out a mnute ago), any H.264 decoder 
> complaint to 
> > a certain profile and level must be able to handle all bitstreams 
> > conforming to that profile and level. That is, a hard limit 
> on frame 
> > rate (regardless of spatial resoltuion) does not apply to 
> > still I don't see a reason to define the new parameter specifically 
> > for H.264.
> > 
> > BR, YK
> >

---

From: Roni Even <[email-masked]>
Date: Mon, 20 Sep 2010 20:03:20 +0200

Hi YK,
This is true but he table in annex A does not allow a receiver in a video
conferencing to ask for a specific image (size, or frame rate) this is
required by video conferencing systems where you may want to negotiate a
specific picture parameters like resolution so the receiver will not need to
scale the picture for example. 
All these parameter depends on the capability of the encoder to prepare such
image which is not required by H.264 since it is up to the receiver to be
able to process any stream within the level limits.

Roni

> Sent: Monday, September 20, 2010 5:13 PM
> To: 'Roni Even'; 'Tom Kristensen'

---

From: Roni Even <[email-masked]>
Date: Mon, 20 Sep 2010 20:06:17 +0200

Hi Tom,
There is a new work in mmusic in
http://tools.ietf.org/html/draft-ietf-mmusic-image-attributes-06 which may
be better suited since it will allow a generic parameter which is not H.264
specific. This is an extensible framework.
Roni Even
As individual

> Sent: Tuesday, September 14, 2010 3:01 PM
> To: IETF AVT WG
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 5451 (20100914) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.eset.com
> 
> 
> 
> 
> __________ Information from ESET NOD32 Antivirus, version of virus
> signature database 5451 (20100914) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.eset.com
>

---

From: Ye-Kui Wang <[email-masked]>
Date: Mon, 20 Sep 2010 14:22:59 -0400

Hi Roni,

Right. For specific image size, as you know, it was discussed earlier and
the conclusion was to use the generic image attribute defined in MMUSIC. For
frame rate, the situation is similar and I think a generic parameter instead
of an H.264 specific parameter should be used. 

BR, YK 

> Sent: Monday, September 20, 2010 2:03 PM
> To: 'Ye-Kui Wang'; 'Tom Kristensen'
> This is true but he table in annex A does not allow a 
> receiver in a video conferencing to ask for a specific image 
> (size, or frame rate) this is required by video conferencing 
> systems where you may want to negotiate a specific picture 
> parameters like resolution so the receiver will not need to 
> All these parameter depends on the capability of the encoder 
> to prepare such image which is not required by H.264 since it 
> is up to the receiver to be able to process any stream within 
> the level limits.
> 
> Roni
> 
> > > Table A.1 in H.264 defines the levels. There is a definition of 
> > > maximum MaxMBPS, MaxFS and MaxBR. There is not definition 
> of what is 
> > > the maximum frame rate, I assume that this can be achieved by 
> > > limiting the MaxMBPS and MaxFS parameters that are 
> defined in  H.241 
> > > and 3984bis. This is what table
> > > > > fps. In such cases we are stuck with no way of signalling a 
> > > > > H.264 specific max frame rate, as in an SDP video 
> media section 
> > > > > motivation for adding max-fps with the same semantics to ease 
> > > > > the gatewaying of SIP and H.323.
> > > > >
> > > > > -- Tom
> > > > >
> > > >
> > > >
> > > > Audio/Video Transport Working Group [email-masked]

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Mon, 20 Sep 2010 22:36:15 -0500

I think the most compelling reason for max-fps in SDP is for H.323/H.241
interworking. 

Extending a=imageattr to include frame rate seems inappropriate since it
is an image (not video) attribute as currently defined. It is also not
possible to answer with the attribute if the offer did not include it,
which can break interworking depending on the call direction.

Using a=framerate works when all payloads have a common limit, but not
if they have different limits. H.264 often has different limits than
other codecs (e.g. H.263) due to higher computational complexity in
software implementations or greater hardware acceleration support in
hardware implementations. Perhaps we should extend a=framerate to
include payload-specific bindings like a=imageattr. But it still seems
best to follow H.241 MaxFPS with H.264-specific max-fps in SDP.

I also think we need to be practical about "conforming to the [H.264]
standard".

H.264 allows frame rates up to 172 fps. For example, level 1.3, which is
widely used for CIF 30 fps, technically allows SQCIF 172 fps. How many
implementations that advertise level 1.3 in SDP support 172 fps?

It also allows aspect ratios as wide / narrow as 8:1 / 1:8. For example,
level 4, which is widely used for 1080p 30 fps, technically allows
4096x512 or 512x4096 30 fps. How many implementations that advertise
level 4 in SDP support 4K horizontal and vertical resolution?

Most if not all real-world implementations don't truly conform to these
limits. The standard specifies unreasonable limits for common
applications because it did not want to impose more restrictive limits
that might constrain the application space (despite being more
appropriate for common applications). So we should be careful about
assuming implementations are "conforming", since most if not all are
not.

Regards,
Mo

-----Original Message-----
Sent: Monday, September 20, 2010 2:23 PM

Hi Roni,

Right. For specific image size, as you know, it was discussed earlier
and
the conclusion was to use the generic image attribute defined in MMUSIC.
For
frame rate, the situation is similar and I think a generic parameter
instead
of an H.264 specific parameter should be used. 

BR, YK 

-----Original Message-----
From: Roni Even [mailto:[email-masked]] 
Sent: Monday, September 20, 2010 2:03 PM
To: 'Ye-Kui Wang'; 'Tom Kristensen'
Cc: 'IETF AVT WG'; 'Malcolm Walters'
Subject: RE: [AVT] I-D 

Hi YK,

Roni

-----Original Message-----

Right, this is my piont as well. A decoder must be able to 
handle all 
possible combinations of spatial resolution and frame rate 
(under the 
constraints by MaxMBPS and MaxFS) to claim itself as 
conforming to the 
standard.

BR, YK

-----Original Message-----
From: Roni Even [mailto:[email-masked]]
Sent: Monday, September 20, 2010 11:01 AM
To: 'Tom Kristensen'
Cc: 'IETF AVT WG'; 'Malcolm Walters'; 'Ye-Kui Wang'
Subject: RE: [AVT] I-D

Hi,

Roni Even
As individual

-----Original Message-----

Tom,

a certain profile and level must be able to handle all 
bitstreams 

BR, YK

-----Original Message-----
From: Tom Kristensen [mailto:[email-masked]]
Sent: Thursday, September 16, 2010 8:20 AM
To: Ye-Kui Wang
Cc: 'Tom Kristensen'; 'IETF AVT WG'; 'Malcolm Walters'
Subject: Re: [AVT] I-D

Yes, this interpretation is valid. The text added in Section
5 of the draft
(http://tools.ietf.org/html/draft-kristensen-avt-rtp-h241param
-00#section-5)
will then be changed to something along the lines of:

   'The "a=framerate" limitation allows a global maximum
   frame rate to be specified. This limit applies to all
   codecs. In cases where both "a=framerate" and 
codec specific
   limits are specified, the maximum frame rate that 
can be used
   for a codec is the minimum of the values specified globally
   and specifically for a codec.'

Then imagine an upcoming H.265, VC-2, VC-666 or 
whatever video 
standard becomes available and typical frame rates rise
to above 60
fps. In such cases we are stuck with no way of signalling a 
H.264 specific max frame rate, as in an SDP video 
media section 
"a=framerate" will then be 120 fps or something and your 
existing
H.264 implementation might just cope with 60 fps or 
even below.

As shown in the text H.261 and H.263 have a way to signal
a maximum
frame rate - explicit or implicit - today. We propose 
the same 
mechanism for H.264.

Another interesting fact is the following note added in
RFC3984 (and
-bis) for the max-br parameter:

           'Informative note: This parameter was 
added primarily
           to complement a similar codepoint in the ITU-T
           Recommendation H.245, so as to facilitate signaling
           gateway designs. [..]'

As max-fps is used today in H.323 systems today, there is
a similar
motivation for adding max-fps with the same semantics to ease 
the gatewaying of SIP and H.323.

-- Tom

---

From: Ye-Kui Wang <[email-masked]>
Date: Tue, 21 Sep 2010 10:05:14 -0400

> Extending a=imageattr to include frame rate seems 
> inappropriate since it is an image (not video) attribute as 
> currently defined. It is also not possible to answer with the 
> attribute if the offer did not include it, which can break 
> interworking depending on the call direction.

If you read the draft, even just the abstract (copied below for
convenience), you will know that the image attribute is not just about
image. Furthermore, if you think that there is any problem regarding the
image attribute itself, your comments would be welcome to improve that
draft. 

   This document proposes a new generic session set up attribute to make
   it possible to negotiate different image attributes such as image
   size.  A possible use case is to make it possible for a low-end hand-
   held terminal to display video without the need to rescale the image,
   something that may consume large amounts of memory and processing
   power.  The draft also helps to maintain an optimal bitrate for video
   as only the image size that is desired by the receiver is
   transmitted.
 
> Using a=framerate works when all payloads have a common 
> limit, but not if they have different limits. 

Note that the semantics is "maximum frame rate", not "common frame rate". 

> Most if not all real-world implementations don't truly 
> conform to these limits. The standard specifies unreasonable 
> limits for common applications because it did not want to 
> impose more restrictive limits that might constrain the 
> application space (despite being more appropriate for common 
> applications). So we should be careful about assuming 
> implementations are "conforming", since most if not all are not.

I think we should be cautious when making the above statements, which are
pretty bold IMO. There have been good reasons for the H.264/AVC standard to
be specified as such with respect to level definitions. The main reason in
my understanding is that the decoding capability is really expressed in
terms of maximum macroblocks per second (MaxMBPS) and maximum frame size in
number of macroblocks (MaxFS), together with other level definition
parameters found in the H.264/AVC specification. I don't see a single reason
why a decoder can handle 352x288 but cannot handle 704x144. The H.264/AVC
level definitions are for real-world implementations and are practical. If
you think otherwise, please name some video decoders that are claimed
conforming but actually they cannot handle all conforming bitstreams (due to
different spatial resolutions and/or frame rates). 

BR, YK

> From: Mo Zanaty (mzanaty) [mailto:[email-masked]] 
> Sent: Monday, September 20, 2010 11:36 PM
> To: Ye-Kui Wang; Roni Even; Tom Kristensen; Malcolm Walters
> Cc: IETF AVT WG
> Subject: RE: [AVT] I-D 
> I think the most compelling reason for max-fps in SDP is for 
> H.323/H.241 interworking. 
> 
> Extending a=imageattr to include frame rate seems 
> inappropriate since it is an image (not video) attribute as 
> currently defined. It is also not possible to answer with the 
> attribute if the offer did not include it, which can break 
> interworking depending on the call direction.
> 
> Using a=framerate works when all payloads have a common 
> limit, but not if they have different limits. H.264 often has 
> different limits than other codecs (e.g. H.263) due to higher 
> computational complexity in software implementations or 
> greater hardware acceleration support in hardware 
> implementations. Perhaps we should extend a=framerate to 
> include payload-specific bindings like a=imageattr. But it 
> still seems best to follow H.241 MaxFPS with H.264-specific 
> max-fps in SDP.
> 
> I also think we need to be practical about "conforming to the 
> [H.264] standard".
> 
> H.264 allows frame rates up to 172 fps. For example, level 
> 1.3, which is widely used for CIF 30 fps, technically allows 
> SQCIF 172 fps. How many implementations that advertise level 
> 1.3 in SDP support 172 fps?
> 
> It also allows aspect ratios as wide / narrow as 8:1 / 1:8. 
> For example, level 4, which is widely used for 1080p 30 fps, 
> technically allows
> 4096x512 or 512x4096 30 fps. How many implementations that 
> advertise level 4 in SDP support 4K horizontal and vertical 
> resolution?
> 
> Most if not all real-world implementations don't truly 
> conform to these limits. The standard specifies unreasonable 
> limits for common applications because it did not want to 
> impose more restrictive limits that might constrain the 
> application space (despite being more appropriate for common 
> applications). So we should be careful about assuming 
> implementations are "conforming", since most if not all are not.
> 
> Regards,
> Mo
> 
> Right. For specific image size, as you know, it was discussed 
> earlier and the conclusion was to use the generic image 
> attribute defined in MMUSIC.
> For
> frame rate, the situation is similar and I think a generic 
> parameter instead of an H.264 specific parameter should be used. 
> 
> BR, YK 
> 
> handle all possible combinations of spatial resolution and 
> frame rate (under the constraints by MaxMBPS and MaxFS) to 
> claim itself as conforming to the standard.
> 
> BR, YK
> 
> attribute in cases where different payload types have 
> different max frame rates, as in that case the value of 
> a=framerate can (and should) be equal to the max of the max 
> frame rates.
> 
> signalling a
> H.264 specific max frame rate, as in an SDP video media 
> section "a=framerate" will then be 120 fps or something and 
> your existing

---

From: Roni Even <[email-masked]>
Date: Tue, 21 Sep 2010 23:54:50 +0200

YK,
I think that the real claim is not that the decoder cannot handle
information which is within the level but that it will be better if the
received information will not be scaled down or frames dropped in order to
reduce traffic on the network and processing (battery) resources. I think
that for your concern the draft should say that this will work if the
encoder can supply the required frame rate but it is not support the
specific frame rate to be compliant. Such statement is also part of the
picture geometry receiver parameters for H.264.
I see the need to allow receiver to have similar functionality to the ones
in H.263 which allows to specify the frame rate with relation to specific
image size.
I was just wondering if it is not time to have such general functionality
since it is relevant to other video codecs but I do not have a specific
suggestion for it, the current proposal is for a parameter carried in fmt so
it is per payload type number

Roni

> From: Ye-Kui Wang [mailto:[email-masked]]
> Sent: Tuesday, September 21, 2010 4:05 PM
> To: 'Mo Zanaty (mzanaty)'; 'Roni Even'; 'Tom Kristensen'; 'Malcolm
> Walters'
> Cc: 'IETF AVT WG'
> Subject: RE: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt
> 
>

---

From: "Malcolm Walters" <[email-masked]>
Date: Mon, 20 Sep 2010 16:46:07 +0200

The goal of this was to improve interoperability and user experience
without going to the full complexity of signalling individual frame
rates and resolutions. Please remember that conformation to the
standards varies between implementations. Also a conforming
implementation could receive 60fps (or higher) but only deliver frames
for further processing/display at 30fps (or lower). This would conform,
since the bitstream would be handled, but wouldn't be an ideal case.
Given that existing implementations don't have this parameter (and maybe
don't support the a=framerate), there will always be a need to support
higher framerates than optimum. 

In terms of interworking between H.323 and SIP, ideally any parameter in
one should be mapped onto one in the other in a simple unambiguous
manner. Although the a=framerate exists for SIP, there is no mapping of
this onto a parameter for H.323. Similarly there is now this parameter
in the H.323 world and having the same in SIP would greatly improve this
situation.

Although I don't know of anything that can do H.263 at 60fps and H.264
only at lower resolutions, there is no guarantee that is the case.
However there are other codecs in use, and in the future many more will
be written. Having limits per codec gives much more flexibility than
having only a global value. I'd be keen to ensure all future codec
support had maximum resolution, frame rate and bitrate information
included per codec as well as global codec-independent versions of
these.

Malcolm

-----Original Message-----
From: Ye-Kui Wang [mailto:[email-masked]] 
Sent: 20 September 2010 15:07
To: Tom Kristensen
Cc: 'Tom Kristensen'; 'IETF AVT WG'; Malcolm Walters
Subject: RE: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt

Tom, 

BR, YK

---

From: "Malcolm Walters" <[email-masked]>
Date: Mon, 20 Sep 2010 17:17:37 +0200

Actually it doesn't. All it defines is the largest frame size and, via
MaxMBPS, the frame rate that can be achieved at that resolution. It
doesn't say anything about what frame rates can be achieved at lower
resolutions. Assuming that MaxMBPS is the only restriction, means
advertising support for 720p30 also means you support CIF at 270 fps.
Currently there is no way so say that you'd rather get anything less.
The new MaxFPS was designed to allow you to do so.

Malcolm

-----Original Message-----
From: Roni Even [mailto:[email-masked]] 
Sent: 20 September 2010 16:01
To: Tom Kristensen
Cc: 'IETF AVT WG'; Malcolm Walters; 'Ye-Kui Wang'
Subject: RE: [AVT] I-D Action:draft-kristensen-avt-rtp-h241param-00.txt

Hi,
Table A.1 in H.264 defines the levels. There is a definition of maximum
MaxMBPS, MaxFS and MaxBR. There is not definition of what is the maximum
frame rate, I assume that this can be achieved by limiting the MaxMBPS
and
MaxFS parameters that are defined in  H.241 and 3984bis. This is what
table
A.6 in H.264 provides so why the extra parameter.

Roni Even
As individual

> Good argument about H.245/H.323. However, as I mentioned in my
previous
> still I don't see a reason to define the new parameter specifically
for

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Sep 2010 00:10:50 +0200

Hi Malcolm,
I do not think that interoperability is a reason if you want to interoperate
with a feature that is not compliant. According to H.264 a receiver must
support any stream that is within the level and present it, the encoder is
just required to send a stream within the level definition. Now if the
document would say that this is an optimization for reducing traffic on the
network and battery resource and it will work only if the encoder can
support it, I think that it will be clearer.
Roni Even 

> Malcolm Walters
> Sent: Monday, September 20, 2010 4:46 PM

---

From: Tom Kristensen <[email-masked]>
Date: Wed, 22 Sep 2010 14:36:34 +0200

Roni,
you were referring to MaxMBPS, MaxFS (and MaxBR) as sufficient. Pasting 
in the paragraph from our draft motivating why this is not good enough 
in our opinion:

  The existing max-mbps and max-fs [I-D.ietf-avt-rtp-rfc3984bis]
  parameters underspecify the video stream and can lead to wasted
  bandwidth and processing resources.  For example, a decoder
  signalling support for w1080p30 (243000 MBPS and 8100 MB) implicitly
  allows the sending of w720p60 (216000 MBPS and 3600 MB).  Similarly,
  if a decoder signalled support for w720p30 (108000 MBPS and 3600 MB)
  the encoder could choose to send w480p60, w448p60 or even wCIF at 120
  frames per second.

-- Tom

---

From: Tom Kristensen <[email-masked]>
Date: Wed, 22 Sep 2010 14:41:56 +0200

> specific. This is an extensible framework.

I'm well aware of this draft, and have followed the discussions and 
draft versions of it. Anyway - and along the lines argued by Mo Zanaty 
in another message in this mail thread, we would rather like to use the 
max-fps as specified in H.241 (for H.323 systems) and as described in 
the draft.

-- Tom

[snip]

---

From: Tom Kristensen <[email-masked]>
Date: Wed, 22 Sep 2010 14:49:52 +0200

> support it, I think that it will be clearer.

Indeed. Good feedback. From the discussion on the list I clearly can see 
the motivation in the draft based on interoperability arguments, trigger 
discussions on H.264 stream conformance and implementations not 
following the standard.

We will return to the draft carpentry workshop and rework and polish the 
draft as suggested.

Also, the too wordy description on H.261 and H.263 details will be 
shortened to get a better S/N ratio in the draft!

-- Tom

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Sep 2010 15:12:39 +0200

Hi Tom,
This will be good. I also suggest that the example in the offer answer
section  will show how you specify different frame rates for different
resolutions.

I think that the text " Any encoder that understands the parameter
   semantics shall constrain the frame rate to rates up to that
   specified.  A receiver should have the ability to process video from
   a sender that does not understand this parameter."
Tries to address the issue that H.264 compliant encoders can send anything
within the level and that the parameter is a preference of the decoder of
the encoder can do it

Roni

> Sent: Wednesday, September 22, 2010 2:50 PM
> To: Roni Even
> Cc: 'Malcolm Walters'; 'Ye-Kui Wang'; 'IETF AVT WG'