Subject: RE: [AVT] Clock skew and RTP

From: "Mike Marchywka" <[email-masked]>
Date: Sat, 19 Oct 2002 19:51:10 -0400

Multiple real-time clocks can be  annoying. There is always a cost benefit issue
to consider. Someone suggested dropping or adding silence data once
clock descrepancies are detected. This strikes me as kind of artifact-prone but
my solution ( using Fourier interpolation to change the sample rate after syncing
with a PLL )
would be much too expensive for a lot of applications. However, the more
general issue of time base synchronization can not always be resolved by
making steps in time ( adding or deleting a block of make-up time ).
Oddly enough, the opposite problem of undesired synchronization can show up
in some systems by mechanisms normally thought of
as too weak to matter ( this is kind of cool:  http://www.physics.gatech.edu/schatz/clocks.html  )

I'm still not sure about terminology here, however. I thought clock skew
was an instantanious difference between two media streams ( say audio and video)
that are sent at the same rate as opposed to the issue here of having two 
clocks with different scale factors  ( one is a little faster than the other).

-----Original Message-----
From: Chuck Harrison [mailto:[email-masked]]
Sent: Friday, October 18, 2002 12:25 PM
To: Dominique Fober
Cc: [email-masked]
Subject: Re: [AVT] Clock skew and RTP

Dominique, all,

Dominique Fober wrote:
> 
> Thanks a lot for your replies. It answers my question: resuming the thread, it appears that RTP provides a mean to detect the clock skew to NTP synchronized hosts via the SR RTCP packets, but don't include (itself or using a dedicated payload) an explicit solution.
> In fact, I was not looking for a concrete solution but rather trying to clarify this point. An interesting thing in your feedback is also to get different points of view and practices concerning the problem (including contextual solutions). Additional feedback in this domain is welcome (if there is) and notably concerning video (may the problem be considered as equivalent in the audio and video domains).

I was looking at this a year or so ago and felt that it would be
useful to develop specific tools to control skew. I have attached
a draft I produced at the time, but it was a one-man project which
definitely would need more participation to go anywhere.

FWIW it seems that the current "professional quality" audio
systems like CobraNet and Gibson Magic operate at layer 2
Ethernet and stay away from IP entirely. However I saw a vendor
at IBC this year (Aaton, of Grenoble) offering sync-over-IP for
film-to-video transfers; no details.

Cheers,
  Chuck Harrison
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Dave Singer <[email-masked]>
Date: Sun, 20 Oct 2002 20:55:01 +0800

I think you have to think about all the clocks that may be involved 
here.  There may be as many as 3 on each end:  the NTP clock, the 
audio sampling clock, and the video sampling clock.

a) The audio sampling clock on both transmitter and receiver may be 
free-running with respect to the NTP clock on that system (and 
usually is).

b) The video clock usually isn't;  most people seem to drive video 
frame rates from the same clock as NTP, but they might take monitor 
refresh, which wouldn't be sync'd to whatever clock they are using 
for NTP.

c) The source and destination NTP clocks may also drift with respect 
to each other.

Note that NTP is only the format of the clocks in RTCP;  it's not the 
required source (which is just as well, for systems which don't have 
the ability to sync over a network, such as an RTP camera).  And even 
if NTP is used, many systems correct only daily or thereabouts, and 
free-run in between.

Generally, there is no assurance that an RTP packet was transmitted 
at its 'RTP timestamp' (because of traffic smoothing).  However, the 
spec. does require that the RTCP sender reports be 'at' their NTP 
timestamp.  A long low-pass filter on these to remove network jitter 
can help you track the source clock.  Buffer fullness might provide 
such a filter, expect that source transmission rates etc. can vary 
(i.e. your buffer may fill when the playout is easy and consuming few 
bytes, and the source decides to send ahead keeping the network full).

Likewise the sender reports tell you when the source audio sampling 
is not quite what it claims.  You can compare the NTP and RTP 
differences and see if the sample count is right for the elapsed 
interval.

Then there is the question of what you do once you know the sender 
clock is running faster than you expect (or slower).  Unless you are 
lucky and there are frequent silence intervals (common in telephony 
but rare in streaming), whereupon you can adjust during silence, you 
can amalgamate the two drift rates together and then adjust your own 
audio playout rate (either in your hardware or sample rate converter) 
to cope.  You might think that this introduces a pitch shift, but of 
course this isn't true;  the source *claimed* to be sampling at say 
22.050 on his clock, but in real physical time that is a different 
rate on your clock.  Actually, you are avoiding pitch shift.

You have to keep doing this.  Temperature or other effects cause 
cheap clocks to wander.  Generally it seems better to drive playout 
from the audio clock, as minor variations in video frame timing are 
imperceptible, whereas audio drop-outs (or attempted frame overlap) 
are not.
-- 
David Singer
Apple Computer/QuickTime
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt