Subject: [AVT] SVC open issues - sprop-spatial-resolution

From: "Even, Roni" <[email-masked]>
Date: Sun, 13 Apr 2008 14:46:48 +0300

Hi,

I am not sure what the requirements for this parameter are.

There is no generic resolution parameter defined. There are specific ones in H.263 and H.261 and they list the supported resolution in relations to the frame rate.

In RFC 3984 the receiver capabilities max-mbps and max-fs allows the receiver to signal that it supports higher macro blocks per second or frame size in macro blocks than the signaled level according to table A-1.

The SAR parameter from draft-ietf-avt-rtp-h264-params-01 allows the receiver to signal the sample aspect ratio that it can receive without distortion.

Note that H.264 receiver MSUT be able to scale and picture that is within its receive capability (Level + override parameters)

There is currently working going on in H.241 to enable a receiver to request specific video mode. The plan is to approve it by the end of this month at the ITU SG16 meeting. It will include a sender capability that will supply the scaling capabilities of the sender. 

Roni Even

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: <[email-masked]>
Date: Sun, 13 Apr 2008 23:29:24 +0300

Hi Roni,
 
This parameter was intended for use in offer/answer when spatial
scalability is in use. The resolution of each offer should be included
in the m line in the SDP for each possible offer. Then the answerer
indicates which of the offers it would like to receive. The max-mbps and
max-fs cannot be used for this purpose due to two reasons. The first
reason is that these parameters are in units of macroblocks per second
or macroblocks, not indicative of the spatial resolution. The second
reason is that these parameters can only be used to indicate decoder
capabilities, not stream property. The SAR parameter from
draft-ietf-avt-rtp-h264-params-01 does not indicate spatial resolution,
either. 
 
Therefore, I don't see a solution other than defining this parameter.
However, as noted in the SVC draft, this parameter is not specific to
SVC, it should be defined somewhere else as a generic SDP parameter that
applies to other visual media type. 
 
BR, YK

________________________________

	From: [email-masked] [mailto:[email-masked]] On
Behalf Of ext Even, Roni
	Sent: Sunday, April 13, 2008 2:47 PM
	To: [email-masked]
	Subject: [AVT] SVC open issues - sprop-spatial-resolution

	Hi,

	There is no generic resolution parameter defined. There are
specific ones in H.263 and H.261 and they list the supported resolution
in relations to the frame rate.

	In RFC 3984 the receiver capabilities max-mbps and max-fs allows
the receiver to signal that it supports higher macro blocks per second
or frame size in macro blocks than the signaled level according to table
A-1.

	The SAR parameter from draft-ietf-avt-rtp-h264-params-01 allows
the receiver to signal the sample aspect ratio that it can receive
without distortion.

	Note that H.264 receiver MSUT be able to scale and picture that
is within its receive capability (Level + override parameters)

	There is currently working going on in H.241 to enable a
receiver to request specific video mode. The plan is to approve it by
the end of this month at the ITU SG16 meeting. It will include a sender
capability that will supply the scaling capabilities of the sender. 

	Roni Even

---

From: "Even, Roni" <[email-masked]>
Date: Mon, 14 Apr 2008 09:25:02 +0300

YK,

My poor understanding of SVC is that just saying what is the resolution
of the stream in the specific m-line is not enough since you may need it
if there are other streams that depend on it which is part of the MMUSIC
draft on dependencies.

I think that the whole issue of describing the layer should be part of
that work.

This is not a general SDP parameter; this is why it was not required
before. It is not a capability but a description of the current stream
which is not needed in RFC 3984 since the information is part of the
streams itself.

BTW: Eventually you will need a way to allow the receiver to define the
layering it wants for example for MCUs to build layouts. This is
discussed now for H.241 for a single layer.

Roni

From: [email-masked] [mailto:[email-masked]] 
Sent: Sunday, April 13, 2008 11:29 PM
To: Even, Roni; [email-masked]
Subject: RE: [AVT] SVC open issues - sprop-spatial-resolution

Hi Roni,

BR, YK

	Hi,

	 

	Roni Even

---

From: <[email-masked]>
Date: Mon, 14 Apr 2008 11:30:16 +0300

Roni,
 
Let me try to explain with an example. Assume that a streaming server
offers a scalable bitstream containing three operation points with
different resolutions, e.g. QCIF, CIF and 4CIF. In the SDP, it should
include multiple m lines, each operation point should be associated with
one m line. Layer dependency is inidcated according to the MMUSIC
decoding dependency draft. Beside layer dependency information, for each
of the operation point, other important information such as bitrate,
frame rate and spatial resolution should also be included. My poor
understanding to SDP is that it provides means to indicated bitrate,
frame rate but not spatial resolution - and I don't why SDP does not
include this parameter.
 
With RFC 3984 it is possible to indicate the spatial resolution of a
stream using the sequence parameter set included in one OPTIONAL media
type parameter sprop-parameter-sets, but there is no other means as I
know. However, in SVC, this sprop-parameter-sets for the 4CIF stream in
the example above may contain sequence parameters associated with
different spatial resolutions. Furthermore, it is not convenient to dig
out the spatial resolution values buried in the variable-length coded
parameter sets in sprop-parameter-sets. 
 
In clusion, I found no convenient way of making a negotiation of an
operation point associated with a certain spatia resolution. That's why
sprop-spatial-resolution was added in the draft. I think I am not along
- see the email sent to the AVT reflector by Jaehwan Kim on 21 March. 
 
Could you please provide a clear solution for this issue, such that the
sprop-spatial-resolution parameter can go?
 
BR, YK

	From: ext Even, Roni [mailto:[email-masked]] 
	Sent: Monday, April 14, 2008 9:25 AM
	To: Wang Ye-Kui (Nokia-NRC/Tampere); [email-masked]
	Subject: RE: [AVT] SVC open issues - sprop-spatial-resolution

	YK,

	My poor understanding of SVC is that just saying what is the
resolution of the stream in the specific m-line is not enough since you
may need it if there are other streams that depend on it which is part
of the MMUSIC draft on dependencies.

	I think that the whole issue of describing the layer should be
part of that work.

	This is not a general SDP parameter; this is why it was not
required before. It is not a capability but a description of the current
stream which is not needed in RFC 3984 since the information is part of
the streams itself.

	BTW: Eventually you will need a way to allow the receiver to
define the layering it wants for example for MCUs to build layouts. This
is discussed now for H.241 for a single layer.

	Roni

	Hi Roni,

	Therefore, I don't see a solution other than defining this
parameter. However, as noted in the SVC draft, this parameter is not
specific to SVC, it should be defined somewhere else as a generic SDP
parameter that applies to other visual media type. 

	BR, YK

		From: [email-masked] [mailto:[email-masked]]
On Behalf Of ext Even, Roni
		Sent: Sunday, April 13, 2008 2:47 PM
		To: [email-masked]
		Subject: [AVT] SVC open issues -
sprop-spatial-resolution

		Hi,

		I am not sure what the requirements for this parameter
are.

		There is no generic resolution parameter defined. There
are specific ones in H.263 and H.261 and they list the supported
resolution in relations to the frame rate.

		In RFC 3984 the receiver capabilities max-mbps and
max-fs allows the receiver to signal that it supports higher macro
blocks per second or frame size in macro blocks than the signaled level
according to table A-1.

		The SAR parameter from draft-ietf-avt-rtp-h264-params-01
allows the receiver to signal the sample aspect ratio that it can
receive without distortion.

		Note that H.264 receiver MSUT be able to scale and
picture that is within its receive capability (Level + override
parameters)

		 

		Roni Even

---

From: "Even, Roni" <[email-masked]>
Date: Mon, 14 Apr 2008 11:44:14 +0300

YK,

What you are describing is an information parameter and not a mode for
negotiation. You will also have problem using the framework parameter
since you want to specify the exact frame rate of the layer while the
parameter is a maximum value and can be used to negotiate the maximum
frame rate of the stream. So you will need specific parameters.

What I was also saying is that there are other requirements for such
parameters like for an MCU who wants the EP to send a CIF and QCIF layer
in order to build two different layouts.

Roni

From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, April 14, 2008 11:30 AM
To: Even, Roni; [email-masked]
Subject: RE: [AVT] SVC open issues - sprop-spatial-resolution

Roni,

BR, YK

	YK,

	Roni

	Hi Roni,

	BR, YK

		Hi,

		 

		Roni Even

---

From: Randell Jesup <[email-masked]>
Date: Mon, 14 Apr 2008 10:33:25 -0400

Everyone interested in this discussion should review the last
time we discussed this in 2004 (note - the thread index links seem to take
you to 2007; prev and next work):

http://www.ietf.org/mail-archive/web/mmusic/current/msg03137.html
and
http://www.ietf.org/mail-archive/web/avt/current/msg04871.html

Note: the discussion goes on for a while.

Also, Roni's suggestion from 2002:
http://www.ietf.org/mail-archive/web/mmusic/current/msg00649.html

Also, from RFC 4566:
      a=framerate:<frame rate>

         This gives the maximum video frame rate in frames/sec.  It is
         intended as a recommendation for the encoding of video data.
         Decimal representations of fractional values using the notation
         "<integer>.<fraction>" are allowed.  It is a media-level
         attribute, defined only for video media, and it is not
         dependent on charset.

Note that this "is intended as a recommendation for the encoding".
I.e. that does not say "you cannot encode at a higher rate"; it says "I'd
like you to encode at this rate (or lower)".  Also, this is media-level, as
it says, so it doesn't help with codec-specific issues; it maps better to
output-device ability (or ability of whatever codec offered is fastest).

"Even, Roni" <[email-masked]> writes:
>There is no generic resolution parameter defined. There are specific ones
>in H.263 and H.261 and they list the supported resolution in relations to
>the frame rate. 

A generic solution would be better, or a specific solution that could be
generalized later.  See previous discussions.  I should note some devices
use weird resolutions, like 352x240 (kinda 1/2 CIF, 1/2 QVGA) or unusual
layouts like 144x176 (portrait mode).  :-)

>In RFC 3984 the receiver capabilities max-mbps and max-fs allows the
>receiver to signal that it supports higher macro blocks per second or
>frame size in macro blocks than the signaled level according to table
>A-1. 

Those can define decoder ability, but not display ability.  Note that
a level 3 decoder could decode QCIF (99 macroblocks) at very high frame
rates, but typically could not display it any faster than the refresh rate
of the physical display device (30/50/60Hz, typically), so higher frame
rates would be wasted.

Maximum display ability is better specified by a=framerate, per above, on a
media level.

I should note that a device which can support max-fs of X and max-mbps of Y
cannot necessarily support them at the same time.  At max-fs, the decoder
might need to use an expensive output scaler, or to decode a maximum
framesize it might have to kick out some data from cache, hurting the
max-mbps.  (For example, at SQCIF the current frame and reference frame(s)
might all fit in L2 or L3 cache, enabling a very high max-mbps, while at
VGA they may not fit, slowing it down a lot.)

How much that matters in practice is another question - I suspect most
don't run that close to the limit.

>frame rate but not spatial resolution - and I don't why SDP does not

It's been discussed (per above), but never handled on a generic level.

>parameter sets in sprop-parameter-sets. 

Exactly; parsing them out of the parameter sets is a non-starter.
 
-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)