Subject: Re: [MMUSIC] Maximum non-scaled image size

From: Teodora Guenkova-Luy <[email-masked]>
Date: Mon, 20 Dec 2004 11:33:21 +0100

Henning Schulzrinne wrote:
 >Do any video payload formats specify, as SDP format attributes or
 >otherwise, the maximum image size supported natively (i.e., without
 >downscaling)?
-------------------------------------------------------------------------
Dear All,

The ITU-T standards (e.g. H.261, H.263, H.264, etc.) explicitly specify 
image sizes for the codecs. H.261 and H.263 specify image sizes 
according to ITU CIF standard in form of (height x width) resolution. 
H.264 also specifies max image sizes per used profile level in form of 
max bits per image, but without specifying aspect ratios. From the MPEG 
standards only MPEG-1 specifies max image size of 352x240 height x width 
resolution. All the other Video MPEGs are with arbitrary image sizes.  
Another possibility to specify image sizes are the VESA standards from 
www.vesa.org (e.g. VESA's VGA BIOS Extensions).  
Note, that some standards for video specify frame-rate, frame-size and 
colour-quality dependencies. Hence, using some standard definitions 
might influence all of these 3 parameters, i.e. the colour-quality 
(number of colours per image) might also influence the amount of network 
data for the video streams.
Note, that frame-size can be specified in 3 different manners:
- (height x width) resolution in (pixel x pixel)
- standard - i.e. CIF or VESA
- max number of bits + aspect ratio - e.g. commonly used aspect ratios 
are height: width = 4:3, 5:4, 11:9, 16:9, 16:10. These aspect ratios 
stem also from CIF and VESA
For the complete and unique video performance specification with SDP in 
case of arbitrary video image specification one needs to have 
specifications for frame-rate, frame-size and colour-quality.
Best Regards,
    Teodora Guenkova-Luy

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Teodora Guenkova-Luy <[email-masked]>
Date: Mon, 20 Dec 2004 19:51:28 +0100

Dear Henning Schulzrinne,
For some codecs the framerate is also a fix parameter of the codec 
definition (e.g. the ITU-T H.26x codecs profiles and levels) and
for other codecs (e.g. MPEG-4) it is a generic parameter, which should 
be defined using descriptions of the media like in SDP.
The framerate is the parameter of biggest influence for the network 
traffic and IMHO that is why nobody has considered the framesize
and the colour quality as generic parameter in SDP up to now. But in 
order to specify performance for codecs with arbitrary definitions
of framesize and the colour quality, you need their media definition in 
SDP (or even better in XML format in SDPng) as well.

With respect to framesize, I would rather put the framesize information 
as one line and would rather specify the following for SDP:

framesize-type = "a=framesize:" sizetype SP width SP height
sizetype = ciftype | vesatype | "arbitrary"

Where the "ciftype" can be described considering the inputs of the 
following table:
CIF Format        Resolution in pixels (width x height)
SQCIF               128 × 96
QCIF                 176 × 144
CIF                    352 × 288
4CIF                  704 × 576
9CIF                  1056 × 864
16CIF                1408 × 1152
TV                     720 × 576
SIF                    352 × 240
SIF (NTSC)      360 × 240
SIF (PAL)         360 × 288

and "vesatype" (AR is Aspect Ratio):

VESA Format    width    height    AR-16:10    AR-16:9    AR-4:3    
non-square pixel
QUXGA-Wide    3820    2400    x           
WUXGA            1920    1200    x           
WSXGA+            1680    1050    x           
WXGA+
  
WXGA                1440    900    x           
                            1280    854    x                          
                              3:2
                            1280    800    x           
                            1280    768    x                           
                             5:3
1080i/1080p        1920    1080                        x       
720i/720p            1280    720                          x       
QUXGA              3200    2400                                        x   
QXGA                 2048    1536                                        
x   
UXGA                1600    1200                                         
x   
SXGA+              1400    1050                                         
x   
SXGA                1280    1024                                         
x                    5:4
XGA                   1024    768                                        
  x   
SVGA                 800    600                                        
    x   
VGA                   640    480                                        
    x   
QVGA                320    240                                        
    x   

If specifying a "ciftype" or "vesatype" the SDP "width" and "height" 
parameters could be optional, e.g.
a=framesize:4CIF

If specifying an "arbitrary" framesize then the  "width" and "height" 
parameters are mandatory and shall be expressed in pixel, e.g.
a=framesize:arbitrary 300 200

Hope this helps,
Teodora Guenkova-Luy

Henning Schulzrinne wrote:

> Thanks for all the explanation. It seems a bit asymmetric that 
> framerate is specified as a generic a= parameter in SDP, while 
> resolution is specified, if at all, on a per-codec basis. Would it 
> make sense to have an 'a=width' and 'a=height' parameter as well?
>
> Teodora Guenkova-Luy wrote:
>
>> The ITU-T standards (e.g. H.261, H.263, H.264, etc.) explicitly 
>> specify image sizes for the codecs. H.261 and H.263 specify image 
>> sizes according to ITU CIF standard in form of (height x width) 
>> resolution. H.264 also specifies max image sizes per used profile 
>> level in form of max bits per image, but without specifying aspect 
>> ratios. From the MPEG standards only MPEG-1 specifies max image size 
>> of 352x240 height x width resolution. All the other Video MPEGs are 
>> with arbitrary image sizes.  Another possibility to specify image 
>> sizes are the VESA standards from www.vesa.org (e.g. VESA's VGA BIOS 
>> Extensions).  Note, that some standards for video specify frame-rate, 
>> frame-size and colour-quality dependencies. Hence, using some 
>> standard definitions might influence all of these 3 parameters, i.e. 
>> the colour-quality (number of colours per image) might also influence 
>> the amount of network data for the video streams.
>> - max number of bits + aspect ratio - e.g. commonly used aspect 
>> ratios are height: width = 4:3, 5:4, 11:9, 16:9, 16:10. These aspect 
>> ratios stem also from CIF and VESA
>> For the complete and unique video performance specification with SDP 
>> in case of arbitrary video image specification one needs to have 

-- 
-----------------------------------------
Teodora Guenkova-Luy

Dept. Distributed Systems, University of Ulm, 
Oberer Eselsberg, 89069 Ulm, Germany

Tel.: +49 (0731) 502-4148 
FAX: +49 (0731) 502-4142

e-Mail: [email-masked]

Home Page: http://www-vs.informatik.uni-ulm.de/~Guenkova/ 

-----------------------------------------

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 22 Dec 2004 21:19:42 +0100

Hi,

I would like to point out that 3GPP has defined in TS 26.234 a SDP 
attribute a=framesize. It has the following definition:

The following media level SDP field is defined for PSS:

- "a=framesize:<payload type number> <width>-<height>"

This gives the largest video frame size of H.263 streams.
The frame size field in SDP is needed by the client in order to properly 
allocate frame buffer memory.  For MPEG-4 Visual streams, the frame size 
shall be extracted from the "config" information in the SDP. For H.264 
(AVC) streams, the frame size shall be extracted from the 
sprop-parameters-sets information in the SDP. For H.263 streams, a PSS 
server shall include the "a=framesize" field at the media level for each 
stream in SDP, and a PSS client should interpret this field, if present. 
Clients should be ready to receive SDP descriptions without this attribute.

If this attribute is present, the frame size parameters shall exactly 
match the largest frame size defined in the video stream. The width and 
height values shall be expressed in pixels.

However, to my understanding, this attribute is not what Henning is 
seeking. I only like to warn you at an early stage about the potential 
name conflict. I am sorry that I haven't had time yet to write the SDP 
registration draft for the PSS specification.

But on the point Henning is asking about, in the 3GPP Packet-based 
Streaming Service, the servers can use UAProf to determine the clients 
screen resolution.

Cheers

Magnus

-- 

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 26 Dec 2004 21:20:04 -0500

One basic question is whether this shouldn't be defined for all video 
codecs, rather than having each codec do this slightly differently.

Once this is clearer, we can then decide whether this is one parameter 
or possibly more. So far, all variations, whether motivated by frame 
buffer size or display size, seem to have the same effect: limit what 
the receiver is willing to receive or indicate what the sender will 
send, where the former seems somewhat more important.

Magnus, I'm not sure where you perceive the difference in semantics to 
my earlier description.

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 27 Dec 2004 19:25:56 +0100

Hi Henning,

See my comments below. I have restructured your paragraphs to try to 
answer your question as well as possible.

 > Magnus, I'm not sure where you perceive the difference in semantics to

Based on the following Sentence:
"Do any video payload formats specify, as SDP format attributes or
otherwise, the maximum image size supported natively (i.e., without
downscaling)?"

It talks about native support of image size, which I interpreted to mean 
actual screen size. As the actual amount of macroblock, size of frame 
buffer, that a receiver is capable of decoding is different from what it 
natively can display. Thus I think there is clear reasons to separate 
the capability for decoding from that of displaying. However I don't 
necessarily think that it is appropriate to consider display resolution 
capabilities in SDP, and definitely not in any overlapping parameter. 
There are good reasons to differentiate the two capabilities.

However if your not interested in the terminal display capabilities, but 
rather decoding capabilities, then see below.

> codecs, rather than having each codec do this slightly differently.
The problem is that the video codecs do all provide some profiles and 
levels describing decoding capabilities. However of course the evolution 
of video encoding has resulted in that they are not described using a 
common profile and level description. Also as the profiles and levels 
describe more than simple image sizes, the profile and levels parameter 
would still be needed if a common frame size parameter existed.

So if you simple want to determine the decoders capability to handle a 
screen size, then I think the profile and level system included in the 
H.26x and MPEG codec are quite sufficient.

> send, where the former seems somewhat more important.

I think one needs to make clear separation between frame-rate and 
frame-size. However I do not think that this is sufficient at all. There 
are so many other parameters in relation to decoding capabilities that 
are to be considered. A few of these are: number of reference frames, 
bit-rate according to the buffer model, number of macro blocks possible 
to decode in a certain time interval, codec features.

Cheers

Magnus Westerlund

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 27 Dec 2004 13:33:38 -0500

> There are good reasons to differentiate the two capabilities.

Fully agreed. Among other differences, the display size is likely a 
"soft" limit, i.e., something that the sender can exceed if you don't 
care about wasting bits. The frame buffer size seems more of a hard limit.

> to decode in a certain time interval, codec features.

I'm trying to find a common, useful subset that has high impact on 
either interoperability or efficiency. Clearly, there will always be 
codec-specific parameters, such as the ones you mention. Maximum display 
size and maximum frame buffer size, however, seem pretty universal.