Subject: [AVT] Clock skew and RTP

From: Dominique Fober <[email-masked]>
Date: Fri, 18 Oct 2002 10:40:00 +0200

I would like to know if there is any mecanism and/or payload defined 
to take account of the clock skew problem.  To be more explicit, here is 
a brief description of what I mean by 'clock skew problem':

Clocks on different stations are not running at the same frequency
(we have commonly measured clock deviations up to 1 per 10000 ms).
For real-time transmissions, it results in a problem similar to a 
consumer / producer problem. 
Let's take an example with a sender host A (the producer) and a receiver 
host B (the consumer):
if the consumer consumes the events faster than produced (the B clock is
running faster than the A clock), it will unavoidably lack data in a 
given delay,
if the consumer consumes the events slower than produced, (the B clock is
running slower than the A clock) it will unavoidably accummulate more and 
more data, resulting sooner or later in storage capacity problems.

Real-time streaming multimedia applications are generally buffering 
up to several seconds of data in order to make up for the transport 
latency variations. Therefore side effects of the clock skew aren't
noticeable unless the transmission lasts several hours.

What's the common way to solve this problem (if there is one) ? 
are there any mecanism and/or payload defined in RTP ?

----------------------------------------------
Dominique Fober               <[email-masked]>
----------------------------------------------
GRAME - Centre National de Creation Musicale -
9 rue du Garet  69001 Lyon France
tel:+33 (0)4 720 737 06 fax:+33 (0)4 720 737 01
http://www.grame.fr

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Alan Clark" <[email-masked]>
Date: Fri, 18 Oct 2002 07:09:34 -0400

This is a problem that dates back many years and applies equally to circuit
switched networks (i.e. timing slip).

For bidirectional streams the classic solution is to lock one clock to the
other - typically using one as the timing source and running the other in
loop back mode.

For audio applications, where you may be dependant on non-adjustable sample
rates for playout, the solution may be to measure the rate of clock drift
and then either

(a)inject interpolated samples or
(b)delete a sample then smooth
 - in order to maintain a constant mean playout buffer level.
(c)use a large buffer

Regards

Alan Clark
Telchemy

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Dominique Fober
Sent: Friday, October 18, 2002 4:40 AM
To: [email-masked]
Subject: [AVT] Clock skew and RTP

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 18 Oct 2002 11:02:52 -0400

At least for audio with silence periods (i.e., voice), this problem goes 
away, as it is dealt with during the normal delay adjustment after a 
talkspurt. No special mechanisms are needed.

---

From: Dominique Fober <[email-masked]>
Date: Fri, 18 Oct 2002 17:44:01 +0200

Thanks a lot for your replies. It answers my question: resuming the thread, it appears that RTP provides a mean to detect the clock skew to NTP synchronized hosts via the SR RTCP packets, but don't include (itself or using a dedicated payload) an explicit solution.
In fact, I was not looking for a concrete solution but rather trying to clarify this point. An interesting thing in your feedback is also to get different points of view and practices concerning the problem (including contextual solutions). Additional feedback in this domain is welcome (if there is) and notably concerning video (may the problem be considered as equivalent in the audio and video domains).
df

---

From: Colin Perkins <[email-masked]>
Date: Fri, 18 Oct 2002 12:15:27 -0400

--> Dominique Fober writes:
>Thanks a lot for your replies. It answers my question: resuming the
>thread, it appears that RTP provides a mean to detect the clock skew to
>NTP synchronized hosts via the SR RTCP packets, but don't include (itself
>or using a dedicated payload) an explicit solution.

The mapping in RTCP SR packets is used for lip-synchronization, but is not
needed to estimate clock skew between sender and receiver. The receiver can
estimate skew by looking at the relative offset between the RTP timestamps
in packets it receives, and its local media clock.

Colin

---

From: "Merhdad Abrishami" <[email-masked]>
Date: Fri, 18 Oct 2002 12:14:48 -0400

I also have a question related to this topic, for those who have looked into
the clock accuracy of various vendors. Since local clocks with higher
inaccuracy can contribute to the clock sckew problem, what is an acceptable
(commonly used) clock accuracy (for exmple 50 ppm), used by manufacturers of
Real-time streaming multimedia applications over IP (such as Voice over IP).
Is there an IETF recommendation in this regard?

Thanks,

Mehrdad Abrishami
Principle Engineer
LSI Logic
240-314-1642

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 18 Oct 2002 12:26:45 -0400

We will have a tech report on this subject shortly. With commodity 
workstations, we have observed clock skew as high as 390 ppm. There is 
no IETF recommendation, since the IETF generally does not mandate 
certain performance levels for equipment.

---

From: Chuck Harrison <[email-masked]>
Date: Fri, 18 Oct 2002 09:25:14 -0700

Dominique, all,

> In fact, I was not looking for a concrete solution but rather trying to clarify this point. An interesting thing in your feedback is also to get different points of view and practices concerning th…

I was looking at this a year or so ago and felt that it would be
useful to develop specific tools to control skew. I have attached
a draft I produced at the time, but it was a one-man project which
definitely would need more participation to go anywhere.

FWIW it seems that the current "professional quality" audio
systems like CobraNet and Gibson Magic operate at layer 2
Ethernet and stay away from IP entirely. However I saw a vendor
at IBC this year (Aaton, of Grenoble) offering sync-over-IP for
film-to-video transfers; no details.

Cheers,
  Chuck Harrison

Network Working Group                                        C. Harrison
Internet-Draft                                 Far Field Associates, LLC
Expires: August 30, 2001                                      March 2001

                 Timebase Interlock in RTP Conferences
                    draft-harrison-avt-interlock-01

Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at http://
   www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on August 30, 2001.

Copyright Notice

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

Abstract

   This memo describes a new RTCP packet type, TBM (Timebase
   management).  Packets of this type carry speed control messages and
   buffer status messages between media sources and receivers.  In
   response to these messages, the timebases may speed up or slow down
   relative to absolute time.  In this manner, timebase drift among
   multiple sources and sinks may be eliminated, and transport latencies
   may be adjusted for.  This supports time-coordinated playout of
   multitrack programs from different servers (e.g.  multilingual
   services, audio- and video-mixing) and is suitable for professional
   audiovisual capture, editing, and exhibition.

Harrison                 Expires August 30, 2001                [Page 1]

Internet-Draft             RTP Media Interlock                March 2001

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Scenarios of Use . . . . . . . . . . . . . . . . . . . . . . .  4
   2.1 Remote Television Feed Scenario  . . . . . . . . . . . . . . .  4
   2.2 Multitrack Playout Scenario  . . . . . . . . . . . . . . . . .  4
   3.  Unique Source Identifier . . . . . . . . . . . . . . . . . . .  6
   4.  Timebase Management Messages . . . . . . . . . . . . . . . . .  7
   4.1 TBM speed control message  . . . . . . . . . . . . . . . . . .  8
   4.2 TBM buffer status message  . . . . . . . . . . . . . . . . . . 10
   5.  SDP Parameters . . . . . . . . . . . . . . . . . . . . . . . . 12
   6.  Models for Timebase Speed Adjustment (Tutorial)  . . . . . . . 13
   7.  Security Considerations  . . . . . . . . . . . . . . . . . . . 18
       References . . . . . . . . . . . . . . . . . . . . . . . . . . 19
       Author's Address . . . . . . . . . . . . . . . . . . . . . . . 19
       Full Copyright Statement . . . . . . . . . . . . . . . . . . . 20

Harrison                 Expires August 30, 2001                [Page 2]

Internet-Draft             RTP Media Interlock                March 2001

1. Introduction

   In the course of audiovisual production, it is necessary to combine
   elements that were recorded at various times on various media into a
   well-coordinated program.  When two or more signal sources ("tracks")
   must be combined (e.g.  audio mixing, video cross-fades, soundtrack-
   to-picture "lip sync"), the system must ensure that time
   synchronization is maintained among all the tracks.  The degree of
   required precision varies from tens of milliseconds (general purpose
   lip-sync) to tens of microseconds (musical audio) to a few
   nanoseconds (broadcast video).

   Traditional audiovisual systems are "hard real time": signals occur
   synchronously on all signal wires.  When multiple signal sources
   exist, they are synchronized to high precision by "genlock" (master
   sync generator controls everything) or "interlock" (source A slaved
   to source B).  In contrast, internet-based audiovisual systems are
   "soft real time": signals experience jitter and are not synchronous
   on the wire; receiver buffers smooth out the delay variations and
   permit a steady output signal to be reconstructed.

   In a conventional RTP [1] conference, each source is "live" and
   contains an independent free-running timebase.  These timebases
   naturally drift relative to one another.  While there are certain
   quality compromises in mixing together several non-synchronized
   sources at the receiving terminal, this mode is quite effective for
   live conference applications.  In an interactive teleconference,
   maintaining low latency is important.

   To the contrary, if coordinated prerecorded material is to be played
   out from two separate sources, it is essential that the timebases of
   the sources be phase locked.  Any relative drift between the clock
   rates of the two sources will eventually accumulate to the point that
   receiver buffers will overflow or starve and synchronization is lost.
   To prevent relative drift, at least one of the source timebases may
   speed up or slow down in a controlled fashion.  This memo describes
   methods for exerting that control.

   Source timebase control may serve to cancel long-term speed drift, to
   compensate for transport latencies, or both.  The practices described
   in this memo are intended for all of these circumstances.

Harrison                 Expires August 30, 2001                [Page 3]

Internet-Draft             RTP Media Interlock                March 2001

2. Scenarios of Use

   The timebase management packets defined in this memo may be used in
   combination with the RTP Timing Information Stream payload TIS [2] to
   achieve the precision stream synchronization required for
   professional applications, as described in another internet-draft
   [3].

2.1 Remote Television Feed Scenario

   In this example we consider a video signal -- perhaps from a special
   event or newsgathering location -- being brought back to a television
   studio, over an IP connection, for broadcast.  The television studio
   maintains a master sync generator ("house sync") running at the US
   standard frequency of 29.97 frames per second, to which the entire
   studio plant is synchronized.  If the remote camera were to operate
   in a "free running" mode, its time base would undoubtedly drift
   slightly relative to the studio's house sync and frames would have to
   be dropped or duplicated during the broadcast.

   Instead, the camera timebase is arranged to respond to RTCP commands
   which cause it to advance or retard very slightly to remain locked to
   house sync.  The receiving application in the studio maintains a
   buffer: packet data arrives, with jitter, from the remote camera and
   departs at a steady "house sync" rate to the studio gear.  If the
   buffer gets too full, a "slow down" message is sent via RTCP to the
   remote camera; if the buffer gets near empty, a "speed up" message is
   sent.

   If the camera is integrated with a computer terminal, this timebase
   adjustment may be handled within the camera control application.  In
   the case that a commercial camera is being used, the same effect is
   achieved when the RTCP commands control a video output signal, which
   carries a "color black" signal to the genlock input of the camera
   itself.

   Any number of remote cameras or other video sources (e.g.  network-
   connected video storage units) may be controlled by the studio in
   this fashion.  This mode, in which the receiving location provides
   the master timing reference, may be called "pull" operation, or
   "remote genlock".

2.2 Multitrack Playout Scenario

   An audiovisual program is logically constructed of several parallel
   "tracks", for example: video, left audio, and right audio.  Although
   it is not now common, we envision that in the future there will be an
   increasing need to compose programs from parallel tracks which are

Harrison                 Expires August 30, 2001                [Page 4]

Internet-Draft             RTP Media Interlock                March 2001

   located on separate servers.  For example, we can imagine an
   educational lecture program of cinema criticism in which the movie
   content and the lecture-hall recording are separately owned and
   served by independent providers.  In some cases, maintaining separate
   servers for multilingual audio and video tracks may have technical
   merit, even on conventional material.

   This scenario must support one-to-many multicast, so a "pull" model
   from the entire set of receivers is inappropriate.  The two or more
   sources may be coordinated by a master sync approach or by a slave
   approach.  In the first approach, a monitoring application "tunes in"
   to the conference and monitors the timestamps being issued by the
   sources.  The monitor compares these timestamps with its own internal
   timebase (perhaps based on NTP or GPS time) and issues "speed up" or
   "slow down" RTCP commands to the individual sources.  By this means,
   both sources are aligned to master sync, and relative drift between
   them is eliminated.  The receiving applications' buffers are only
   required to deal with network jitter, as they should.

   In the "slave" approach, one of the program streams is designated as
   master, and the other sources slave their timebases to it.  This mode
   does not require "speed up" and "slow down" messages to be exchanged,
   provided that each signal source is smart enough to monitor the
   stream issued from the master, and is configured to do so.  However,
   the "slave" approach can also be implemented as a special case of the
   "master sync" approach described in the previous paragraph.  In this
   case, the monitor application uses the master source as its timebase
   reference, and only issues speed adjustment commands to the slave
   sources.

Harrison                 Expires August 30, 2001                [Page 5]

Internet-Draft             RTP Media Interlock                March 2001

3. Unique Source Identifier

   A timebase management message makes reference to one or more specific
   sources within a multimedia session ("conference").  Often each of
   these sources participates in a separate RTP session for a particular
   media type (e.g.  audio, video).  The TBM message includes a field
   which points unambiguously to each media source.

   While SSRC identifiers are guaranteed to be unique within a single
   RTP session, the same SSRC value may legally appear in several
   concurrent RTP sessions.  The <SSRC, transport-address> tuple
   uniquely identifies a source, and would be a candidate for the unique
   source identifier field.

   In a general network situation, the transport address can be re-
   mapped between subsections of the network.  Network Address
   Translators (NATs) perform this function in IP networks.  In the most
   general case, RTP sessions can be exported to non-IP systems.  A
   receiving application has no reliable method to know whether the
   transport-address it uses is the same transport-address visible to
   the originator of the TBM message.  Therefore it is difficult for the
   issuer of a TBM message to generate an <SSRC, transport-address>
   identifier which will be useful throughout the complete network.

Harrison                 Expires August 30, 2001                [Page 6]

Internet-Draft             RTP Media Interlock                March 2001

4. Timebase Management Messages

   A Timebase Management packet consists of a header followed by zero or
   more timebase management messages.  There are two defined families of
   TBM messages: speed command and buffer status.  Most commonly, buffer
   status messages are originated by receiver (sink) applications and
   processed by a control application; this results in speed command
   messages to source timebases.  However, applications may use these
   tools in more sophisticated ways, including speed control messages
   directed to receivers and buffer status messages from sources.

   The structure of a TBM  packet is shown below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|    CC   |   PT=TBM=205  |             length            | header
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         SSRC of sender                        |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 SSRC_1 (SSRC of first source)                 |  TBM
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ message
   |R|  T  |V=0|msg len|s.exponent |s.         mantissa            |   1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    timestamp (optional)                       |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 SSRC_2 (SSRC of second source)                |  TBM
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ message
   |R|  T  |V=0|msg len|s.exponent |s.         mantissa            |   2
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    timestamp (optional)                       |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   :                          . . .                                :
   :                                                               :
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                   SSRC_n (SSRC of last source)                |  TBM
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ message
   |R|  T  |V=0|msg len|s.exponent |s.         mantissa            |   n
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    timestamp (optional)                       |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

   Figure 1.  TBM Packet.

   The first two words of the TBM packet are a header containing the
   following fields:

   version (V): 2 bits This field identifies the version of RTP.  The

Harrison                 Expires August 30, 2001                [Page 7]

Internet-Draft             RTP Media Interlock                March 2001

      version defined by this specification is two (2).

   padding (P): 1 bit If the padding bit is set, the packet contains one
      or more additional padding octets at the end which are not part of
      the payload.  The last octet of the padding contains a count of
      how many padding octets should be ignored.  Padding is not
      required by the format specified in this memo, so this bit is 0.

   command count (CC): 5 bits The number of timebase adjustment commands
      included in this packet.  Zero is valid but useless.

   packet type (PT): 8 bits Contains the constant 205 to identify this
      as an RTCP TBM packet.

   length: 16 bits The length of this RTCP packet in 32-bit words minus
      one, including the header and any padding.

   SSRC: 32 bits The synchronization source identifier for the
      originator of this TBM packet.

   The header is followed by a sequence of command blocks, each targeted
   to the timebase of a particular source.  A command block may be a
   speed control message or a buffer status message.  These are
   described further below.

4.1 TBM speed control message

   The structure of a TBM  speed control message is shown below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        SSRC of source                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |R| T=0 |V=0|msg len|s.exponent |s.         mantissa            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    timestamp (optional)                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 2.  TBM speed control message (T=0).

   The fields in the speed control message are:

   SSRC: 32 bits The synchronization source identifier for the timebase
      which is the target of this command.

   reserved (R): 1 bit This field is reserved and should be transmitted
      as 0.

Harrison                 Expires August 30, 2001                [Page 8]

Internet-Draft             RTP Media Interlock                March 2001

   message type (T): 3 bits This field identifies the TBM message type.
      The value 0 indicates a speed control message.

   version (V): 2 bits This field identifies the version of TBM.  The
      version defined by this specification is zero (0).

   msg len: 4 bits This field indicates the number of 32-bit words
      following the word containing this field.  The value 1 is used if
      a 32-bit source timestamp is included in the message.  The value 0
      is used if no timestamp is included.  No other values are
      pertinent to this version of the TBM message format, but larger
      values may be needed in future versions.

   speed correction: 22 bits (6 bit exponent + 16 bit mantissa) This
      field is a fractional correction to the existing speed of the
      specified source timebase.  This parameter is interpreted as
      (mantissa)*(2**exponent).  The mantissa is interpreted as a 2's
      complement binary number with the radix point to the left of the
      most significant bit; thus it takes on values from -1 to (1 -
      2**(-15)).  The exponent is interpreted as a 2's complement binary
      integer, taking on values from -32 to +31.  A parameter value of 0
      indicates no change is desired.  A value of +0.01 commands that
      the timebase increase its speed by 1%.

   timestamp: 32 bits This is the media timestamp, from the targeted
      SSRC, at which the correction was computed.  This field is
      optional, and may be suppressed if RTCP bandwidth is restricted.
      However, in some configurations improved synchronization precision
      can be obtained if the timestamp information is transmitted.

Harrison                 Expires August 30, 2001                [Page 9]

Internet-Draft             RTP Media Interlock                March 2001

4.2 TBM buffer status message

   The structure of a TBM buffer status message is shown below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        SSRC of source                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |R|  T  |V=0|msg len|s.exponent |s.         mantissa            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    timestamp (optional)                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Figure 2.  TBM buffer status message (T=1..3).

   The structure of the buffer status message is identical to the speed
   control message.  The fields have the same meanings, except as
   described below:

   message type (T): 3 bits This field identifies the TBM message type.
      The values 1 thru 3 indicate a buffer status message.

      T=1: Buffer capacity message, issued periodically, and if buffer
         size is reallocated.  Parameter is total buffer size in kbytes
         (1024 bytes).

      T=2: Buffer is at least half full.  Parameter is amount of
         contiguous unused buffer capacity in kbytes.

      T=3: Buffer less than half full.  Parameter is amount of data
         remaining in buffer in kbytes, including "holes" (see below).

   buffer parameter: 22 bits (6 bit exponent + 16 bit mantissa) This
      field is describes an aspect of the receive buffer condition,
      depending on the value of the message type code T (see above).
      This parameter is interpreted as (mantissa)*(2**exponent).  The
      mantissa is interpreted as a 2's complement binary number with the
      radix point to the left of the most significant bit; thus it takes
      on values from -1 to (1 - 2**(-15)).  However, in buffer status
      messages, the mantissa is always positive (sign=0).  The exponent
      is interpreted as a 2's complement binary integer, taking on
      values from -32 to +31.

   A buffer can have "holes" in it due to out-of-order packet arrival.
   Thus it may be "more than half full" even though some media packets
   are unavailable for delivery from the buffer.  Applicable
   retransmission and error-concealment practices are out of the scope

Harrison                 Expires August 30, 2001               [Page 10]

Internet-Draft             RTP Media Interlock                March 2001

   of this memo.

   A receiver may need to execute a "cut" from one source to another
   (e.g.  for commercial insertion).  If the receiver uses a common
   buffer, it must independently track the data from the "cut in" source
   and the "out" source.  In this case it may send independent (and
   apparently contradictory) buffer status messages to the two different
   sources.  The application may send "no space available" messages to
   the "in" source in advance of the scheduled cutover.

   Buffer status reports unnecessarily occupy network bandwidth if the
   network delay and jitter conditions are such that risk of under- and
   over-run is small.  Therefore applications should not transmit buffer
   status messages without reason.  Applications may provide periodic
   buffer status messages if the reporting period is long enough to make
   minimal impact on RTCP bandwidth.

Harrison                 Expires August 30, 2001               [Page 11]

Internet-Draft             RTP Media Interlock                March 2001

5. SDP Parameters

   The fact that a source responds to TBM packets may be published to
   other conference participants by means of the Session Description
   Protocol (SDP) [4].  For example:

        m=audio 1234 RTP/AVP 95          : dynamically assigned audio payload
        a=rtpmap:95 L16/48000/2          : stereo 48kHz
        a=tbm-speed-response             : this source responds to TBM speed cmds

Harrison                 Expires August 30, 2001               [Page 12]

Internet-Draft             RTP Media Interlock                March 2001

6. Models for Timebase Speed Adjustment (Tutorial)

   In this section we discuss several models for maintaining timebase
   synchronization in a distributed network.  These models are
   illustrative examples.  This memo establishes no requirements for
   implementation of specific control architectures.

   Figure 3 shows how the timebase functions in a simple single-stream
   RTP conference using the conventional "push" mode.  The receiver
   ("sink") timebase follows the source timebase by speeding up when the
   receiver buffer runs too full and slowing down when the receiver
   buffer runs too empty.

        +---------+      +---------+      +---------+         +---------+
        |  data   |      | network |      |  rcvr   |         |  data   |
   ====>|  pump   |=====>|  delay  |=====>| buffer  |========>|  pump   |===>
        |         |      |         |      |         |<--------|         |
        +---------+      +---------+      +---------+         +---------+
             ^                                 |                   ^
             |                          "phase |                   |
        +---------+                      error"|  +--------+  +---------+
        | source  |                            |  |control |  |  sink   |
        |timebase |                            +->| algo-  |->|timebase |
        |(freerun)|                               | rithm  |  |         |
        +---------+                               +--------+  +---------+

                    data flow ======>
            control influence ------>

   Figure 3.  Conventional "push" timebase control with receive-side
   PLL.

   Note that the lower arrow connecting the rcvr buffer and the
   receiver's data pump points leftward: the speed of the data pump
   *controls* the occupancy of the buffer, even though the data flow is
   from buffer to pump.  The control arrows illustrate that the receiver
   timebase constitutes a feedback loop; in fact, it is a "phase locked
   loop" or PLL.  It sometimes requires considerable effort to obtain
   the optimum "tuning" (i.e.  the best control algorithm) for a PLL
   system.

   In conventional operation, as illustrated in Fig.  3, the timing PLL
   is entirely contained in the receiving application.  Therefore, no
   network protocols are required to support it, and it is left
   unmentioned by the RTP standards.  The implementation of a
   satisfactory control algorithm is up to the receiving application.

Harrison                 Expires August 30, 2001               [Page 13]

Internet-Draft             RTP Media Interlock                March 2001

   As noted in the introduction to this memo, more sophisticated
   applications require that the source timebase be controlled, rather
   than being permitted to free-run.  The simplest configuration
   illustrating this is the single-stream "pull" arrangement of Figure
   4.

          +---------+      +---------+      +---------+     +---------+
          |  data   |      | network |      |  rcvr   |     |  data   |
   ======>|  pump   |=====>|  delay  |=====>| buffer  |====>|  pump   |===>
          |         |----->|         |----->|         |     |         |
          +---------+      +---------+      +---------+     +---------+
               ^                                 |               ^
               |                                 |"phase         |
          +---------+   speed  +---------+       | error"   +---------+
          | source  |  command | control |       |          |  sink   |
          |timebase |<---------|  algo-  |<------+          |timebase |
          |         |          |  rithm  |                  |         |
          +---------+          +---------+                  +---------+

                    data flow ======>
            control influence ------>

   Figure 4.  "Pull" timebase control.

   The PLL feedback loop is again apparent.  However, in this case, the
   "phase error" information in the receiver must eventually pass back
   to the source timebase over a network protocol in order to close the
   loop.  The TBM messages perform this function.  Being a feedback
   system, the PLL requires attention to stability and closed-loop
   response issues.  A poor selection of the control algorithm may
   result in sluggish correction, or overcorrection, or even unstable
   behavior.

   In different implementations, the "control algorithm" block may be
   located in different places.  If the control block is implemented as
   part of the source application, then TBM buffer status messages will
   close the loop and no TBM speed control messages are required.  On
   the other hand, if the control block is implemented as part of the
   receiver, TBM speed control commands close the loop and no buffer
   status messages are needed.

   However, both of these configurations are rather limited in
   capability.  In most cases, the control algorithm is implemented as a
   separate "monitor" application which receives buffer status messages,
   analyzes them, and produces speed command messages.  Only this
   architecture is capable of properly controlling a multi-source,
   multi-sink configuration (Figure 5).

Harrison                 Expires August 30, 2001               [Page 14]

Internet-Draft             RTP Media Interlock                March 2001

      +--------+                    ----                   +----------+
      | Source |==============>  (         ) =============>| Receiver |
      +--------+<----------   (      (       )     <------ +----------+
                            (      (          )  )
      +--------+           (                      )        +----------+
      | Source |==========> (       Network      ) =======>| Receiver |
      +--------+<-----------  (                )  <------- +----------+
          :                    (     Cloud    )                 :
          :                   (            )    )               :
      +--------+<------------- (          )    ) <-------- +----------+
      | Source |=============>    ----(       ) ==========>| Receiver |
      +--------+                    ^   -----              +----------+
                                    |    |
                                spd |    |
                 +-----------+  ctrl|    |
        Monitor  |  Control  |------+    |buffer
      Application| Algorithm |<----------+ status
                 +-----------+

   Figure 5.  Multi-source, multi-sink timebase control with receiver
   feedback.

   The architecture of figure 5, in principle, can monitor several
   receiver applications, and optimize the source timebases so as to
   eliminate buffer over- or under-run events.  This precision "pull"
   behavior is likely to be useful in combination with a reliable
   transport protocol in small-scale applications such as within a
   recording studio.

   On the other hand, when a large number of receivers are distributed
   over a large unreliable network, a different approach is suitable.
   It is no longer possible to guarantee on-time delivery to all
   receivers, and it is wasteful to send buffer status messages.
   Instead, a "master sync" or "slave" mode, as discussed above under
   "Multitrack Playout Scenario", is applicable.  This is illustrated in
   Figure 6.

Harrison                 Expires August 30, 2001               [Page 15]

Internet-Draft             RTP Media Interlock                March 2001

      +--------+                    ----                   +----------+
      | Source |==============>  (         ) =============>| Receiver |
      +--------+<----------   (      (       )             +----------+
                            (      (          )  )
      +--------+           (                      )        +----------+
      | Source |==========> (       Network      ) =======>| Receiver |
      +--------+<-----------  (                )           +----------+
          :                    (     Cloud    )                 :
          :                   (            )    )               :
      +--------+<------------- (          )    )           +----------+
      | Source |=============>    ---(          ) ========>| Receiver |
      +--------+                    ^ (        )           +----------+
                                    |   (     )
                                spd |     (  ) ===+
                 +-----------+  ctrl|  +-------+  |
                 |  Control  |------+  |"dummy"|<=+
                 | Algorithm |<------- | rcvr  |
                 +-----------+         |buffer |
                                       +-------+
                  Monitor Application

   Figure 6.  Multi-source, multi-sink timebase control with no receiver
   feedback.

   The performance required of the PLL feedback system is distinctly
   different between those applications in which prerecorded material is
   being played out and those involving a live acquisition source.  This
   is apparent when considering the response of the loop to changes in
   the transport delay (latency).

   In the case of a prerecorded source, it is generally possible to
   speed up, i.e.  advance, the playout in order to compensate for an
   increased latency.  Likewise, the playout can be retarded if latency
   decreases.  This function is comparable to flow control used in
   ordinary data transfers, and optimizes the usage of buffer space.
   There is little concern if the data stream plays out from the source
   at a somewhat irregular rate.  In this case, a simple "bang-bang"
   speed control algorithm, which toggles between a slightly fast
   timebase and a slightly slow one, may be adequate.

   In the case of a "live" source, the situation is different.  The
   system may be operating stably with a certain amount of transport
   delay.  If the delay increases, and the increase exceeds the buffer
   capacity, then buffers will underflow and there will be a
   discontinuity in the received signal.  In audio, particularly, this
   type of data loss is very undesirable.  For this reason, many high-
   quality applications will use large data buffers in order to resist
   fluctuations in transport rate, accommodate error-correction delays,

Harrison                 Expires August 30, 2001               [Page 16]

Internet-Draft             RTP Media Interlock                March 2001

   etc.  There is a tradeoff in increased system latency, which is of
   little concern unless interactivity is needed.  In a multi-stream
   "live" system, relative drift between source timebases can cause one
   stream's buffers to drain and eventually underflow.  This can be
   corrected by increasing the sampling rate of that channel.  Such rate
   changes must be carefully executed by a well-filtered speed-control
   algorithm in high-fidelity applications.  This filtered algorithm may
   exist as a separate PLL embedded in the source application, as
   illustrated in Figure 7.

       +---------+      +--------+    +--------+          +--------+   +--------+
       | A-to-D  |      |  xmtr  |    |  data  | network  |  rcvr  |   |  data  |
   ===>|converter|=====>| buffer |===>|  pump  |==.....==>| buffer |==>|  pump  |===>
       |         |----->|        |<---|        |--     -->|        |   |        |
       +---------+      +--------+    +--------+          +--------+   +--------+
            ^                  |          ^                   |             ^
            |                  +--+       |                   |"phase       |
       +---------+   +---------+  |  +--------+  +---------+  | error" +---------+
       | source  |   | control |  |  | source |  | control |  |        |  sink   |
       |timebase |<--|  algo-  |<-+  |timebase|<-|  algo-  |<-+        |timebase |
       |  (A/D)  |   | rithm 2 |     | (xmtr) |  | rithm 1 |           |         |
       +---------+   +---------+     +--------+  +---------+           +---------+

         Highly filtered PLL            Bang-bang PLL

                                data flow ======>
                        control influence ------>

   Figure 7.  "Live" timebase control with source-side buffering and
   local PLL.

Harrison                 Expires August 30, 2001               [Page 17]

Internet-Draft             RTP Media Interlock                March 2001

7. Security Considerations

   The proposals in this memo present few new security considerations.
   It is possible that a defective or malicious application may send
   inconsistent information in TBM packets which would affect the
   performance of a source.  Applications should respond in a safe
   manner to inconsistent timing commands.  It is recommended that
   source applications implement a "hard limit" on the maximum data rate
   at which they will transmit, regardless of the TBM messages they
   receive.

Harrison                 Expires August 30, 2001               [Page 18]

Internet-Draft             RTP Media Interlock                March 2001

References

   [1]  Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson,
        "RTP: A Transport Protocol for Real-Time Applications", RFC
        1889, January 1996.

   [2]  Harrison, C., "RTP Payload Format for Timing Information Stream
        (TIS)", draft-harrison-avt-tis-00 (work in progress), March
        2001.

   [3]  Harrison, C., "Audiovisual Transport with Precision Timing",
        draft-harrison-avt-precision-av-00 (work in progress), February
        2001.

   [4]  Handley, M. and V. Jacobson, "SDP: Session Description
        Protocol", RFC 2327, April 1998.

Author's Address

   Chuck Harrison
   Far Field Associates, LLC
   18815 111th Pl SE
   Snohomish, WA  98290
   US

   Phone: +1 360 863 8340
   EMail: [email-masked]

Harrison                 Expires August 30, 2001               [Page 19]

Internet-Draft             RTP Media Interlock                March 2001

Full Copyright Statement

   Copyright (C) The Internet Society (2001).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

Harrison                 Expires August 30, 2001               [Page 20]

---

From: "Dan Firac" <[email-masked]>
Date: Mon, 21 Oct 2002 09:56:53 +0200

Hi Colin,

See comments below.

----- Original Message -----
From: "Colin Perkins" <[email-masked]>
To: "Dominique Fober" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, October 18, 2002 6:15 PM
Subject: Re: [AVT] Clock skew and RTP

> in packets it receives, and its local media clock.
The problem with your solution is that the receiver measurements (your so
called 'relative offset') contains the network jitter which can disturb any
skew estimation algorithm.
There is some work about skew estimation algorithms, but what is their
performance? and most of all, what is the cost?
I've studied this problem a lot and I found that the most accurate measure
(without network jitter) can be done only by the source which fills the SR
RTCP packet with the two informations(local media clock + local system
clock). In practice, the NTP skew is much less than the media skew, and that
makes this solution more simpler and more accurate than one with skew
estimation.
Anyway, the RTP rfc doesn't say that RTCP SR to be used ONLY for
lip -synchronization (inter-media sinchronization).
This is the RTP timestamp definition:

    RTP timestamp: 32 bits
             Corresponds to the same time as the NTP timestamp (above),
             but in the same units and with the same random offset as
             the RTP timestamps in data packets. This correspondence may
             be used for intra- and inter-media synchronization for
             sources whose NTP timestamps are synchronized, and may be
             used by media-independent receivers to estimate the nominal
             RTP clock frequency. Note that in most cases this timestamp
             will not be equal to the RTP timestamp in any adjacent data
             packet. Rather, it MUST be calculated from the
             corresponding NTP timestamp using the relationship between
             the RTP timestamp counter and real time as maintained by
             periodically checking the wallclock time at a sampling
             instant.

> Colin

Best regards,
Dan.

---

From: Colin Perkins <[email-masked]>
Date: Mon, 21 Oct 2002 10:25:45 -0400

>skew estimation algorithm.

True, the measurements have to be filtered to limit the effects of jitter
and find the long term drift.

>lip -synchronization (inter-media sinchronization).

RTP doesn't define the skew correction algorithm, so you can certainly use
the information in RTCP SR packets if you think it's useful. The tool I'm
most familiar with uses a much simpler algorithm:
  http://www-mice.cs.ucl.ac.uk/multimedia/publications/ICME2000.ps.gz
which seems to work well for voice tools, where there are talk-spurts to
allow adjustments. A continuous streaming application would likely need
something more sophisticated.

Colin