Subject: [MMUSIC] RTSP: Speed and Scale overview text

From: Magnus Westerlund <[email-masked]>
Date: Tue, 02 Dec 2008 13:45:50 +0100

Hi,

Below is an overview text about Speed and Scale. Trying to ensure that
we all agree what Scale and Speed is supposed to do. So please review
and see if you think you understand this. If it matches your
understanding of the concepts and what need you see of either of the
mechanisms. I want to ensure that we are on the same page and do see a
need for both before updating the rest of the text regarding these subjects.

Media Delivery Manipulations

The basic playback functionality of RTSP is to request content for a
particular range to be delivered to the client in a pace that enables
playback as intended by the creator. However, RTSP can also manipulate
how this delivery is done to the client in two ways.

   Scale: The ratio of media content delivered per unit playback time.

   Speed: The ratio of playback time delivered per unit of wall clock
          time.

So both affects the media delivery per time unit. However, they are
manipulating two independent time scales and the effects are possible to
combine.

Scale is used for fast forward or slow motion control as it changes the
amount of content timescale that should be played back per time unit.
Scale > 1.0, means fast forward, e.g. Scale=2.0 results in that 2
seconds of content is played back every second of playback. Scale = 1.0
is the default value that is used if no Scale is specified, i.e.
playback at the contents original rate. Scale values between 0 and 1.0
is providing for slow motion. Scale can be negative to allow for reverse
playback in either regular pace (Scale = -1.0) or fast backwards (Scale
< -1.0) or slow motion backwards (-1.0 < Scale < 0).

In most cases the realization of scale means server side manipulation of
the media to ensure that the client can actually play it back. These
media manipulation and when they are needed are highly media type
dependent. Lets exemplify with two common media types audio and video.

It is very difficult to modify the playback rate of audio. A maximum of
10-30% is possible by changing the pitch-rate of speech. Music goes out
of tune if one tries to manipulate the playback rate by resampling it.
This is a well known problem and audio is commonly muted or played back
in short segments with skips to keep up with the current playback point.

For video is possible to manipulate the number of frames that is
displayed per second. But the rendering capabilities are often limited
to certain frame rates. The decoding, handling capabilities and bitrate
of received encoded content also limits the number of frames that can be
delivered. Therefore when providing fast forward one generally picks a
subset of the frames from the original content to be displayed. However,
the video encoding methods use will commonly limit the possibilities on
which frames that can be chosen and still be decoded by the receiver.

Due to the media restrictions a particular content will commonly be
restricted to a limited set of possible scale ratios. To handle this
correctly, RTSP has mechanism to indicate the supported Scale ratios for
the content. To support aggregated or dynamic content where this may
change during the ongoing session and dependent on the location within
the content a mechanism for updating the media properties and the
current used scale factor exist.

Speed affects how much of the playback timeline that is delivered in a
given wall clock period. The default is Speed = 1 which is to deliver at
the same rate the media is consumed. Speed > 1 means that the receiver
will get content faster than it regularly would consume it. Speed < 1
means that it receives it slower than the regular consumption rate. This
mechanism enables two general functionalities. Client side scale
operations, i.e. the client receives all the frames and makes the
adjustment to the playback locally. The second usage is to control its
buffering of media. By specifying a speed over 1.0 the client can build
up the amount of playback time it has present in its buffers to a level
that is sufficient for its needs.
-- 

Magnus Westerlund

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------
_______________________________________________
mmusic mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mmusic

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 08 Dec 2008 17:34:19 +0100

Hi,

This is an email to trying to attempt to summarize the situation.

I think people are agreeing on the scale definition and that it is a
useful tool in some situations. I will therefore draft a spec solution
that tries to fix the known issues, including a media properties
attribute that carries a list of supported scale values for the media.

When it comes to Speed although people seems to agree on a very high
level on what speed is there is still no conclusion on why it is needed,
if at all. We need more input into why Speed is used and for what.
Without that understanding make clear specification text becomes impossible.

Cheers

Magnus Westerlund