Subject: [AVT] Fwd: I-D Action:draft-perkins-avt-rapid-rtp-sync-03.txt

From: Colin Perkins <[email-masked]>
Date: Mon, 9 Mar 2009 19:43:01 +0000

The changes in this version are primarily editorial; we're aware that  
not all the open issues have been addressed yet. Any further comments  
are welcomed.

Chairs: we'd like this to be considered as an AVT working group  
draft, as indicated privately.

Cheers,
Colin

Begin forwarded message:
> From: [email-masked]
> Date: 9 March 2009 17:15:02 GMT
> To: [email-masked]
> Subject: I-D Action:draft-perkins-avt-rapid-rtp-sync-03.txt
> Reply-To: [email-masked]
>
> A New Internet-Draft is available from the on-line Internet-Drafts  
> directories.
>
> 	Title           : Rapid Synchronisation of RTP Flows
> 	Author(s)       : C. Perkins, T. Schierl
> 	Filename        : draft-perkins-avt-rapid-rtp-sync-03.txt
> 	Pages           : 17
> 	Date            : 2009-03-09
>
> This memo outlines how RTP multimedia sessions are synchronised, and
> discusses how rapidly such synchronisation can occur.  We show that
> most RTP sessions can be synchronised immediately, but that the use
> of video switching multipoint conference units (MCUs) or large source
> specific multicast (SSM) groups can greatly increase the initial
> synchronisation delay.  This increase in delay can be unacceptable to
> some applications that use layered and/or multi-description codecs.
>
> This memo updates the RTP Control Protocol (RTCP) timing rules to
> reduce the initial synchronisation delay for SSM sessions.  A new
> feedback packet is defined for use with the Extended RTP Profile for
> RTCP-based Feedback (RTP/AVPF), allowing video switching MCUs to
> rapidly request resynchronisation.  Two new RTP header extensions are
> defined to allow rapid synchronisation of late joiners, and guarantee
> correct timestamp based decoding order recovery for layered codecs in
> the presence of clock skew.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-perkins-avt-rapid-rtp- 
> sync-03.txt
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> Content-Type: text/plain
> Content-ID: <[email-masked]>
>
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Tue, 10 Mar 2009 10:04:51 -0700

Hi Colin,

I read the draft and I found it informative. The funny thing it shares a
similar title with our draft :)
I wonder whether we can come up with a better, non-colliding name for
any of the drafts. Furthermore, we are also defining control messages (3
for now), one of which is rapid (multicast) synch request (RMS-R).
Similar names can be confusing, IMO.

Others comments:
- FYI: Section 3.1: reg. "next available FMT is 5?" We used 5, 6 and 7
in the our draft. Of course, it is not reserved yet.
- Could the draft also mention about the other useful properties of
having a sender report earlier? I also think a more appropriate name
could be found for this message.
- Section 4.2 says "The frequency of intersing ntp header exs is up to
the sender." Any guidance/constraints on this?
- There are a few inconsistent intra-document references.

Chairs:
I support this draft to become a WG item as it has two good solution for
(timestamp) synchronization for RTP flows.

-acbegen

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Colin Perkins
> Sent: Monday, March 09, 2009 12:43 PM
> To: [email-masked] WG
> Subject: [AVT] Fwd: I-D Action:draft-perkins-avt-rapid-rtp-sync-03.txt
> 
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>

---

From: Ye-Kui Wang <[email-masked]>
Date: Tue, 10 Mar 2009 13:33:49 -0400

I fully agree that using the same term for two very different subjects is
confusing. For example, this has wasted me a bit time when looking for
emails for either subject. 

To me, the term "rapid sync" matches draft-perkins-avt-rapid-rtp-sync-03
better than draft-versteeg-avt-rapid-synchronization-for-rtp-02, as the
former is really about sync of different streams, while the latter is aboud
rapid tunning-in or channel switching. 

BR, YK 

> Behalf Of Ali C. Begen (abegen)
> Sent: Tuesday, March 10, 2009 1:05 PM
> To: Colin Perkins; [email-masked]
> Subject: Re: [AVT] Fwd: I-D 
> Action:draft-perkins-avt-rapid-rtp-sync-03.txt
> 
> Hi Colin,
> 
> I read the draft and I found it informative. The funny thing 
> it shares a similar title with our draft :) I wonder whether 
> we can come up with a better, non-colliding name for any of 
> the drafts. Furthermore, we are also defining control 
> messages (3 for now), one of which is rapid (multicast) synch 
> request (RMS-R).
> - FYI: Section 3.1: reg. "next available FMT is 5?" We used 
> 5, 6 and 7 in the our draft. Of course, it is not reserved yet.
> - Could the draft also mention about the other useful 
> properties of having a sender report earlier? I also think a 
> more appropriate name could be found for this message.
> - Section 4.2 says "The frequency of intersing ntp header exs 
> is up to the sender." Any guidance/constraints on this?
> Behalf Of 
> > Colin Perkins
> > Subject: [AVT] Fwd: I-D 
> Action:draft-perkins-avt-rapid-rtp-sync-03.txt
> > 
> > Chairs: we'd like this to be considered as an AVT working 
> group draft, 
> > as indicated privately.
> > 
> > Cheers,
> > Colin
> > 
> > 
> > 
> > 
> > Begin forwarded message:
> > > Internet-Draft directories: http://www.ietf.org/shadow.html or 
> > > ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> >

---

From: "Roni Even" <[email-masked]>
Date: Wed, 11 Mar 2009 18:25:34 +0200

Hi,
I think that some re-naming will be helpful. At least for the draft names we
can do it if accepted as WG items.

As for FMT numbers. I think that when we have WG documents it will be better
to use different numbers.

Regards
Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Ali C.
Begen (abegen)
Sent: Tuesday, March 10, 2009 7:05 PM
To: Colin Perkins; [email-masked]
Subject: Re: [AVT] Fwd: I-D Action:draft-perkins-avt-rapid-rtp-sync-03.txt

Hi Colin,

Others comments:

Chairs:

-acbegen

---

From: Thomas Schierl <[email-masked]>
Date: Thu, 12 Mar 2009 13:45:42 +0100

Hi Ali,

> Similar names can be confusing, IMO.

Actually, I do not really care about the naming, but I feel that what 
you are doing is more about rapid or fast tune-in than about 
synchronization which I personally connect with timing. Anyway, I would 
not mind to change the name of our draft and/or the message names. But I 
guess that can be "synchronized" once the drafts are accepted as WG items.

> could be found for this message.

As I said before, naming is not the problem. Which useful properties 
should be added here?

> the sender." Any guidance/constraints on this?

Guidelines would be helpful, but we didn't want to constraint this 
approach. I think it is clear the more often you insert them, the 
earlier you can, e.g., start the decoding order recovery. I think this 
is definitely depending on the application. What would you expect here? 
Do you think about guidance with respect to overhead against the time it 
takes to be synced?

> - There are a few inconsistent intra-document references.

Thanks, will check that.

> (timestamp) synchronization for RTP flows.

Best regards,
Thomas

-- 
Thomas Schierl
--------------
Fraunhofer HHI

>>> of video switching multipoint conference units (MCUs) or 
>>>       
>> large source
>>     
>>> synchronisation delay.  This increase in delay can be 
>>>       
>> unacceptable to
>>     
>>> rapidly request resynchronisation.  Two new RTP header 
>>>       
>> extensions are
>>     
>>> defined to allow rapid synchronisation of late joiners, and 
>>>       
>> guarantee
>>     
>>> correct timestamp based decoding order recovery for layered 
>>>       
>> codecs in
>>     

----
Visit us at

OFC 2009 / March 24-26 / San Diego, California / Hall B1, Booth 411
http://www.ofcnfoec.org/

Web 2.0 Expo / March 31 - April 03 / San Francisco, California / Booth 415
http://www.web2expo.com/webexsf2009

---

From: Stefan Döhla <[email-masked]>
Date: Thu, 12 Mar 2009 15:09:14 +0100

Colin, Thomas, all,

the draft is describing the proposed header extension and RTCP SR 
request quite good, however:

Isn't the NTP timestamp generation subject to the same problems as in 
RTCP? I could easily send an RTCP SR as well for every RTP packet within 
a sampling instance of one (in terms of RTP time), i.e. there could be 
an RTP packet and an RTCP SR packet containing the same RTP timestamp if 
I were able to generate them virtually at the same time (just a few ns 
away). Clearly, this is not datarate-efficient nor sensible, but I don't 
see a big difference to the proposed approach.

So what about the source for the NTP time: Should the NTP time be 
filtered at the sender before inclusion into the HDR_EXT? Because if 
not, the NTP timestamp would either be only related to the RTP timeline 
(bad - then we have two different timelines, one via RTCP, the other in 
the HDR_EXT) or be jittery (bad because not instantly useful, though the 
same timeline as RTCP).

An additional comment on the draft is that section 4.3 is quite complex 
and hard to understand, though (I think) correct.

And the last one: What was wrong again with signaling/knowing that two 
streams come from the same source/encoder and would be perfectly aligned 
after reception of one NTP-RTP mapping per stream or even directly 
aligned (same RTP timeline)? I tried to find the answer in the list 
archive, but maybe you can give me a hint.

Cheers
- Stefan

Colin Perkins schrieb:
> Chairs: we'd like this to be considered as an AVT working group draft, 
> as indicated privately.
>
> Cheers,
> Colin
>
>
>
>
> Begin forwarded message:
>> http://www.ietf.org/internet-drafts/draft-perkins-avt-rapid-rtp-sync-03.txt 
>>
>>

-- 
Dipl.-Ing. Stefan Döhla     | Email: [email-masked]
Multimedia Transport        | Phone: +49 (0)9131 776 6042 (!NEW!)
Audio Department            | Fax:   +49 (0)9131 776 6099 (!NEW!)
Fraunhofer IIS              |
Am Wolfmantel 33            |
91058 Erlangen, Germany     | Web:   http://www.iis.fraunhofer.de/amm

---

From: "Ali C. Begen (abegen)" <[email-masked]>
Date: Thu, 12 Mar 2009 09:52:36 -0700

Hi Thomas, 

> As I said before, naming is not the problem. Which useful properties 

I am not sure but one could be regarding the extended RTP seqnums. An SR would enable the receiver to learn the ext seqnum.
 
> Do you think about guidance with respect to overhead against 

That could be one aspect. Others may include whether the receiver has to process all such header extensions, the processing overhead of such extensions, issues with MTU limits... Not sure if all these will make sense, though.

BR,
-acbegen

---

From: [email-masked]
Date: Fri, 13 Mar 2009 12:33:56 +0100

Hi Stefan,

Stefan Döhla wrote:
> RTCP? I could easily send an RTCP SR as well for every RTP packet 
> within a sampling instance of one (in terms of RTP time), i.e. there 
> could be an RTP packet and an RTCP SR packet containing the same RTP 
> timestamp if I were able to generate them virtually at the same time 
> (just a few ns away). Clearly, this is not datarate-efficient nor 
> sensible, but I don't see a big difference to the proposed approach.
>
> not, the NTP timestamp would either be only related to the RTP 
> timeline (bad - then we have two different timelines, one via RTCP, 
> the other in the HDR_EXT) or be jittery (bad because not instantly 
> useful, though the same timeline as RTCP).

Here it is important to use the same NTP timestamp value for the same 
point of synchronization in all the sessions. This is only guaranteed, 
if such header extensions are used for the same sampling time instance 
in all the sessions, i.e. using the same NTP wallclock timestamp. I 
think this is stated in the draft.

>
> An additional comment on the draft is that section 4.3 is quite 
> complex and hard to understand, though (I think) correct.

It is anyway planned to spend some more work on this section. I try to 
improve the text.

> streams come from the same source/encoder and would be perfectly 
> aligned after reception of one NTP-RTP mapping per stream or even 
> directly aligned (same RTP timeline)? I tried to find the answer in 
> the list archive, but maybe you can give me a hint.
>

The main issue was backward compatibility.

Thomas

----
Visit us at

---

From: Stefan Döhla <[email-masked]>
Date: Fri, 13 Mar 2009 13:13:05 +0100

Hi Thomas,

> think this is stated in the draft.
Let me rephrase it a little: If I accidentally receive an RTCP SR with 
the same RTP timestamp as in an RTP packet in either session, would the 
NTP timestamp be the same?

[snip]

---

From: [email-masked]
Date: Fri, 13 Mar 2009 14:16:02 +0100

Stefan Döhla wrote:
>>> Isn't the NTP timestamp generation subject to the same problems as 
>>> in RTCP? I could easily send an RTCP SR as well for every RTP packet 
>> point of synchronization in all the sessions. This is only 
>> guaranteed, if such header extensions are used for the same sampling 
>> time instance in all the sessions, i.e. using the same NTP wallclock 
>> timestamp. I think this is stated in the draft.
> the same RTP timestamp as in an RTP packet in either session, would 
> the NTP timestamp be the same?
For the same RTP timestamp in SR and RTP packet, the NTP timestamp in 
the header extension as well as in the SR shall be the same.

Thomas

----
Visit us at

---

From: Stefan Döhla <[email-masked]>
Date: Fri, 13 Mar 2009 14:39:36 +0100

Hi Thomas,

[email-masked] wrote:
>>>> in RTCP? I could easily send an RTCP SR as well for every RTP 
>>>> packet within a sampling instance of one (in terms of RTP time), 
>>>> i.e. there could be an RTP packet and an RTCP SR packet containing 
>>>> the same RTP timestamp if I were able to generate them virtually at 
>>>> the same time (just a few ns away). Clearly, this is not 
>>>> datarate-efficient nor sensible, but I don't see a big difference 
>>>> to the proposed approach.
>>>>
>>>> filtered at the sender before inclusion into the HDR_EXT? Because 
>>>> if not, the NTP timestamp would either be only related to the RTP 
>>> Here it is important to use the same NTP timestamp value for the 
>>> same point of synchronization in all the sessions. This is only 
>> Let me rephrase it a little: If I accidentally receive an RTCP SR 
>> with the same RTP timestamp as in an RTP packet in either session, 
>> would the NTP timestamp be the same?
Then I think my assumption was correct, that you need to pre-filter the 
NTP timestamps so that they increase roughly linearly - but at least 
monotonically with each according RTP timestamp - otherwise it would be 
possible to align samples from several layers but it is no more 
guaranteed that they have the timestamps in correct order). And in 
addition, the SRs must be filtered then as well to correspond to the 
HDR_EXT timestamps (- what as I understood is a problem in practice 
where NTP timestamps are often way off).

If a stream is re-timestamped by an intermediate device the 
correspondence would be broken if not the HDR_EXT is also adjusted. Also 
this device would have to follow the same rules.

Cheers
- Stefan

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 13 Mar 2009 15:22:59 -0400

Some more comments in addition to those from others posted earlier.

   Section 1:

   Two backwards compatible extensions to the basic
   RTP synchronisation mechanism are proposed:

   o  An enhancement to the Extended RTP Profile for RTCP-based Feedback
      (RTP/AVPF) [2] is defined to allow receivers to request additional
      RTCP SR packets, providing the metadata needed to synchronise RTP
      flows.  This can reduce the synchronisation delay when joining
      sessions with large RTCP reporting intervals, or in the presence
      of packet loss.

   o  Two RTP header extensions are defined, to deliver synchronisation
      metadata in-band with RTP data packets.  These extensions provide
      synchronisation metadata that is aligned with RTP data packets,
      and so eliminate the need to estimate clock-skew between flows
      before synchronisation.  They can also reduce the need to receive
      RTCP SR packets before synchronising flows. 

The two extensions seem to be a bit contradicting to each other. The benefit
of the first is to allow receivers to request more SR packets, while one can
be reduce the need of SR packets. How to use them both? If only one should
be used at a time, when to use which? Are both really needed?

   Section 2.1.2:

        Session| Number of receivers (single sender assumed):
      Bandwidth|  2     3     4     5     10   100   1000  10000
             --+------------------------------------------------
         8 kbps| 2.73  4.10  5.47  5.47  5.47  5.47  5.47  5.47
        16 kbps| 2.50  2.50  2.73  2.73  2.73  2.73  2.73  2.73
        32 kbps| 2.50  2.50  2.50  2.50  2.50  2.50  2.50  2.50
        64 kbps| 2.50  2.50  2.50  2.50  2.50  2.50  2.50  2.50
       128 kbps| 1.41  1.41  1.41  1.41  1.41  1.41  1.41  1.41
       256 kbps| 0.70  0.07  0.07  0.07  0.07  0.07  0.07  0.07
       512 kbps| 0.35  0.35  0.35  0.35  0.35  0.35  0.35  0.35
         1 Mbps| 0.18  0.18  0.18  0.18  0.18  0.18  0.18  0.18
         2 Mbps| 0.09  0.09  0.09  0.09  0.09  0.09  0.09  0.09
         4 Mbps| 0.04  0.04  0.04  0.04  0.04  0.04  0.04  0.04

   Figure 1: Average RTCP Reporting Interval (seconds)

It seems that the number 0.70 for 512 kbps with 2 receivers is typo (of
0.07)? 

   Section 3.2. 

Two variants of RTP header extension are specified, with the difference
being whether to signal the 8 MSBs of the integer part of the NTP timestamp.
I think having either but only one of the two variants, preferrably the one
without the 8 MSBs, is enough. 

BR, YK

> Sent: Monday, March 09, 2009 3:43 PM
> To: [email-masked] WG
> The changes in this version are primarily editorial; we're 
> aware that not all the open issues have been addressed yet. 
> Any further comments are welcomed.
> 
> group draft, as indicated privately.
> 
> Cheers,
> Colin
> 
> 
> 
> 
> Begin forwarded message:
> > most RTP sessions can be synchronised immediately, but that 
> the use of 
> > video switching multipoint conference units (MCUs) or large source

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 13 Mar 2009 15:35:51 -0400

> The main issue was backward compatibility.

I also need some refreshing of my memory. Is the backward compatibility
issue the one like "Timing based synchonization of RTP flows must be based
on RTCP SRs according to RFC 3550", or something else? 

Another point just came to my mind. When we were discussing other possible
mechanims based on RTP header extentions, e.g. by including something like
decoding order numbers, there was an argument that the mechanim MUST work if
the optional RTP header extensions are removed by intermediate network
elements. How does that argument relate to the RTP header extension mechanim
in this draft?

BR, YK

> Behalf Of [email-masked]
> Sent: Friday, March 13, 2009 7:34 AM
> Cc: [email-masked] WG; Colin Perkins
> Here it is important to use the same NTP timestamp value for 
> the same point of synchronization in all the sessions. This 
> is only guaranteed, if such header extensions are used for 
> the same sampling time instance in all the sessions, i.e. 
> using the same NTP wallclock timestamp. I think this is 
> stated in the draft.
> 
> >
> It is anyway planned to spend some more work on this section. 
> I try to improve the text.
> 
> >
> aware that 
> >>> of video switching multipoint conference units (MCUs) or large 
> >>> source specific multicast (SSM) groups can greatly increase the 
> >>> initial synchronisation delay.  This increase in delay can be 
> >>> unacceptable to some applications that use layered and/or 
> multi-description codecs.
> >>>
> >>> rapidly request resynchronisation.  Two new RTP header extensions 
> >>> are defined to allow rapid synchronisation of late joiners, and 
> >>> guarantee correct timestamp based decoding order recovery for 
> >>> layered codecs in the presence of clock skew.
> >>>
> OFC 2009 / March 24-26 / San Diego, California / Hall B1, 
> Booth 411 http://www.ofcnfoec.org/
>

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 13 Mar 2009 15:43:15 -0400

Hi Stefan,

> Then I think my assumption was correct, that you need to 
> pre-filter the NTP timestamps so that they increase roughly 
> linearly - but at least monotonically with each according RTP 
> timestamp - otherwise it would be possible to align samples 
> from several layers but it is no more guaranteed that they 
> have the timestamps in correct order).

I need an example to follow you here :-)

BR, YK 

> Behalf Of Stefan D?hla
> Sent: Friday, March 13, 2009 9:40 AM
> To: [email-masked]
> Cc: Colin Perkins; [email-masked] WG
> Then I think my assumption was correct, that you need to 
> pre-filter the NTP timestamps so that they increase roughly 
> linearly - but at least monotonically with each according RTP 
> timestamp - otherwise it would be possible to align samples 
> from several layers but it is no more guaranteed that they 
> have the timestamps in correct order). And in addition, the 
> SRs must be filtered then as well to correspond to the 
> HDR_EXT timestamps (- what as I understood is a problem in 
> practice where NTP timestamps are often way off).
> 
> correspondence would be broken if not the HDR_EXT is also 
> adjusted. Also this device would have to follow the same rules.
> 
> Cheers
> - Stefan
> >
> > Thomas
> >
> >>
> >> [snip]
> >>
> 
> --

---

From: [email-masked]
Date: Mon, 16 Mar 2009 09:25:25 +0100

> be used at a time, when to use which? Are both really needed?

We think depending on the use case, both can be helpful. For fast 
tune-in A/V sync with having a backward channel, the first method is to 
favor. If exact matching of NTP timestamps is required as for 
timestamp-based decoding order recovery for layered codecs, the second 
is in favor.

>       Bandwidth|  2     3     4     5     10   100   1000  10000

If you mean the 256kbps line, I think yes.

> without the 8 MSBs, is enough. 

This may be correct, although this version is not compliant with ISMA.

Thomas

>>> This memo outlines how RTP multimedia sessions are 
>>>       
>> synchronised, and 
>>     
>>> feedback packet is defined for use with the Extended RTP 
>>>       
>> Profile for 
>>     

----
Visit us at

---

From: [email-masked]
Date: Mon, 16 Mar 2009 09:27:12 +0100

Hi Ye-Kui,

Ye-Kui Wang wrote:
>  
>   
>>> And the last one: What was wrong again with 
>>>       
>> signaling/knowing that two 
>>     

The other point was not to allow receiver implementations which break 
the SR-based approach.

> elements. How does that argument relate to the RTP header extension mechanim

I guess that has been discussed earlier. See, e.g., 
http://www.ietf.org/mail-archive/web/avt/current/mail17.html
There was also more discussion on the list in December 2008.
I think, if there is a middle box which does not know about the new 
layered codec and hence does not know about the new header extensions, 
such device wouldn't do anything reasonable with the layered media 
anyway. So, I cannot see an issue.

Thomas

>>> Isn't the NTP timestamp generation subject to the same 
>>>       
>> problems as in 
>>     
>>> within a sampling instance of one (in terms of RTP time), 
>>>       
>> i.e. there 
>>     
>>> could be an RTP packet and an RTCP SR packet containing the 
>>>       
>> same RTP 
>>     
>>> timestamp if I were able to generate them virtually at the 
>>>       
>> same time 
>>     
>>> filtered at the sender before inclusion into the HDR_EXT? 
>>>       
>> Because if 
>>     
>>> And the last one: What was wrong again with 
>>>       
>> signaling/knowing that two 
>>     
>>>> not all the open issues have been addressed yet. Any 
>>>>         
>> further comments 
>>     
>>>> are welcomed.
>>>>
>>>>> A New Internet-Draft is available from the on-line 
>>>>>           
>> Internet-Drafts 
>>     
>>>>> directories.
>>>>>
>>>>> discusses how rapidly such synchronisation can occur.  We 
>>>>>           
>> show that 
>>     
>>>>> most RTP sessions can be synchronised immediately, but 
>>>>>           
>> that the use 
>>     
>> http://www.ietf.org/internet-drafts/draft-perkins-avt-rapid-rtp-sync
>>     
>>>>> -03.txt
>>>>>
>>>>>

----
Visit us at

---

From: Stefan Döhla <[email-masked]>
Date: Mon, 16 Mar 2009 10:29:22 +0100

Hi Ye-Kui,

> I need an example to follow you here :-)

The example would be related to the bad behaviour of some 
implementations that create quite rough RTCP SRs when it comes to 
timestamp creation. I think Asterisk was a good example of NTP 
timestamps that jump back and forth. If the same bad clock would be used 
to write the NTP timestamp for the HDR_EXT, then you would end up with 
subsequent packets where the RTP timestamp is e.g. monotonically 
increasing but the NTP timestamp is not because there's a high jitter added.

I don't say the HDR_EXT is bad per se (it has other advantages, see the 
ISMA paper) but it would be good if a little more restrictions can be 
put on the sender to ensure that the NTP and RTP timestamps are well 
correlated - though as I said before, this would not break the ability 
of the solution to find the layers for a single point in time.

Cheers
- Stefan
>>>>>> the draft is describing the proposed header extension 
>>>>>>             
>> and RTCP SR 
>>     
>> problems as 
>>>>>> i.e. there could be an RTP packet and an RTCP SR packet 
>>>>>>             
>> containing 
>>     
>>>>>> the same RTP timestamp if I were able to generate them 
>>>>>>             
>> virtually at 
>>     
>>>>>> datarate-efficient nor sensible, but I don't see a big 
>>>>>>             
>> difference 
>>     
>>>>>> So what about the source for the NTP time: Should the 
>>>>>>             
>> NTP time be 
>>     
>>>>>> filtered at the sender before inclusion into the 
>>>>>>             
>> HDR_EXT? Because 
>>     
>>>>>> if not, the NTP timestamp would either be only related 
>>>>>>             
>> to the RTP 
>>     
>>>>>> timeline (bad - then we have two different timelines, 
>>>>>>             
>> one via RTCP, 
>>     
>>>>>> the other in the HDR_EXT) or be jittery (bad because not 
>>>>>>             
>> instantly 
>>     
>>>>> guaranteed, if such header extensions are used for the 
>>>>>           
>> same sampling 
>>     
>>>>> time instance in all the sessions, i.e. using the same 
>>>>>           
>> NTP wallclock 
>>     
>>> For the same RTP timestamp in SR and RTP packet, the NTP 
>>>       
>> timestamp in 
>>

---

From: [email-masked]
Date: Mon, 16 Mar 2009 12:01:15 +0100

Hi Stefan,

Stefan Döhla wrote:
>>>>> the same RTP timestamp if I were able to generate them virtually 
>>>>> at the same time (just a few ns away). Clearly, this is not 
>>>>> timeline (bad - then we have two different timelines, one via 
>>>>> RTCP, the other in the HDR_EXT) or be jittery (bad because not 
>>>>> instantly useful, though the same timeline as RTCP).
>>>>
>>>> guaranteed, if such header extensions are used for the same 
>>>> sampling time instance in all the sessions, i.e. using the same NTP 
>>>> wallclock timestamp. I think this is stated in the draft.
> Then I think my assumption was correct, that you need to pre-filter 
> the NTP timestamps so that they increase roughly linearly - but at 
> least monotonically with each according RTP timestamp - otherwise it 
> would be possible to align samples from several layers but it is no 
> more guaranteed that they have the timestamps in correct order). And 
> in addition, the SRs must be filtered then as well to correspond to 
> the HDR_EXT timestamps (- what as I understood is a problem in 

I agree, if you have such problems with your source wallclock, as we 
discussed intensively before and during the Minneapolis meeting, you 
would have to do so. But RFC 3550 anyway requires for the timestamps to 
be monotonically and linearly increasing. And as also discussed earlier, 
the linearity of the clock may be the problem in practice. I cannot see 
a new point here.

> correspondence would be broken if not the HDR_EXT is also adjusted. 
> Also this device would have to follow the same rules.

I guess the answer would be in the same spirit as other answers to 
backward compatibility with respect to layered codecs: If there is a 
mixer which does not know about the new layered codec and hence does not 
know about the new header extensions, such device wouldn't do anything 
reasonable with the layered media anyway. But this may be a reason for 
requiring the use of the header ext for timestamp based decoding order 
recovery in the layered codec drafts.

Thomas

----
Visit us at

---

From: Stefan Döhla <[email-masked]>
Date: Mon, 16 Mar 2009 12:58:43 +0100

[email-masked] wrote:
>>>>>> Isn't the NTP timestamp generation subject to the same problems 
>>>>>> as in RTCP? I could easily send an RTCP SR as well for every RTP 
>>>>>> containing the same RTP timestamp if I were able to generate them 
>>>>>> virtually at the same time (just a few ns away). Clearly, this is 
>>>>>> not datarate-efficient nor sensible, but I don't see a big 
>>>>>> difference to the proposed approach.
>>>>>>
>>>>> sampling time instance in all the sessions, i.e. using the same 
>>>>> NTP wallclock timestamp. I think this is stated in the draft.
>>> For the same RTP timestamp in SR and RTP packet, the NTP timestamp 
>>> in the header extension as well as in the SR shall be the same.
> would have to do so. But RFC 3550 anyway requires for the timestamps 
> to be monotonically and linearly increasing. And as also discussed 
> earlier, the linearity of the clock may be the problem in practice. I 
> cannot see a new point here.
Okay, I wasn't aware of the discussion at the Minneapolis meeting (that 
I haven't attended). Maybe you can just clarify this a little bit better 
in the draft.
> mixer which does not know about the new layered codec and hence does 
> not know about the new header extensions, such device wouldn't do 
> anything reasonable with the layered media anyway. But this may be a 
> reason for requiring the use of the header ext for timestamp based 
> decoding order recovery in the layered codec drafts.
Okay. If backwards compatibility is not an issue, then everything's 
basically fine and most of my points are void. Considering all this, I 
would also like to see this becoming a WG item.

Cheers
- Stefan

[snip]

---

From: Randell Jesup <[email-masked]>
Date: Mon, 16 Mar 2009 23:49:59 -0400

Stefan Döhla <[email-masked]> writes:
>>> guaranteed that they have the timestamps in correct order).
>>
>The example would be related to the bad behaviour of some implementations
>that create quite rough RTCP SRs when it comes to timestamp creation. I
>think Asterisk was a good example of NTP timestamps that jump back and
>forth. If the same bad clock would be used to write the NTP timestamp for
>the HDR_EXT, then you would end up with subsequent packets where the RTP
>timestamp is e.g. monotonically increasing but the NTP timestamp is not
>because there's a high jitter added.

"jumping around" doesn't even come close to describing what Asterisk does
to RTCP timestamps.  "Virtually random" is closer, or "totally useless".

Sorry, sore point. 1/2 :-)

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Michael A Dolan <[email-masked]>
Date: Mon, 30 Mar 2009 11:01:23 -0700

Those interested in this topic may also find this interesting:

http://www.atsc.org/standards/cs_documents/a153-2009-03-13/S4-132r13-A153-Part-3-Transport.pdf

Section 10.3.

Some differences are:

1. There is one NTP packet stream per service, rather than the 
encoding optimization in the hdr ext;
2. RTCP SR records are emitted "near" video random access points, 
rather than on a prescribed time schedule; and
3. The above works in multicast-only, UDLR networks (e.g. television 
broadcast), rather than requiring a 2-way link.

#2 is interesting since in order to begin a *synchronized* decode 
(video + audio properly correlated), a client must have acquired all of:
a. NTP
b. RTCP SR for each component
c. video reference frame and decoding metadata (i.e. a random access point)
d. audio frame

So, the relative timing of the RTCP SR records to the RTP stream 
content is perhaps more important than the emission period.  Having 
the RTCP SR records arrive faster than the video reference frames 
doesn't help of course.  And even if they are "fast enough", having 
them arrive out of sync with the reference frames delays the service 
acquisition by half the period on average.

Bandwidth utilization (how many bits to devote to RTCP and NTP) can 
arguably be more efficiently managed by the server in most cases, not 
individual client requests for faster RTCP SR packets. And, the 
client would have no knowledge of the intended (video) reference 
frame periods being used by the server, so it would not, in general, 
know what to ask for.

In practice, won't a server that wants to enable faster acquisition 
and sync do things like the above?  I'm concerned that detailed 
requests from clients beyond "please use more bandwidth and enable 
fast sync" won't be as valuable as they seem since in the general 
case the client doesn't know what to ask for exactly.

FYI, the above ATSC document is in a "candidate" state where public 
comments are solicited.  Independent of this thread, if anyone has 
comments on it, I could convey them back to the ATSC.

Regards,

         Mike

At 07:43 PM 3/9/2009 +0000, Colin Perkins wrote:
>>http://www.ietf.org/internet-drafts/draft-perkins-avt-rapid-rtp- sync-03.txt
>>

---

From: [email-masked]
Date: Tue, 31 Mar 2009 09:34:43 +0200

Dear Michael,

Thanks for the comments and the reference to the ATSC M/H.

I guess, what we are looking for in AVT is a solution which also works 
for encoders having non-perfect clocks. There was a long discussion on 
the list, here are two pointers:

http://www.ietf.org/mail-archive/web/avt/current/msg09144.html
http://www.ietf.org/mail-archive/web/avt/current/msg10491.html

ATSC surely can assume to have a perfect clock in the NTP streams, which 
will not be the case on PC systems that may be used as an encoder for 
multi session transmission of a layered codec. This is one major problem 
we intended to solve with the header extension while keeping the whole 
approach as close as possible in line with the existing RTCP SR method.

Best regards,
Thomas

______________________________________________
Thomas Schierl, Dipl.-Ing.
Fraunhofer HHI
Senior Research Engineer
Image Processing Dept. / Multimedia Transport
Einsteinufer 37, 10587 Berlin, Germany
PHONE: +49 30 31002 227 , FAX: +49 30 392 7200
WEB: http://ip.hhi.de , SKYPE: thomas.schierl
EMAIL: [email-masked]

Michael A Dolan wrote:
> individual client requests for faster RTCP SR packets. And, the client 
> would have no knowledge of the intended (video) reference frame 
> periods being used by the server, so it would not, in general, know 
> what to ask for.
>
> fast sync" won't be as valuable as they seem since in the general case 
> the client doesn't know what to ask for exactly.
>

---
Visit us at

NAB SHOW 2009 / April 18-23 / Las Vegas, Nevada USA / South Hall (Upper), Booth SU9624F
http://www.nabshow.com/

---

From: Michael A Dolan <[email-masked]>
Date: Tue, 31 Mar 2009 07:36:49 -0700

Hi Thomas-

Thanks.  Yes, I am familiar with the discussion.

I'm not necessarily pushing the ATSC solution, but offering some 
points to consider, which apply equally to the problem at hand here, I think.

Also, it is important to note some more things about the ATSC text:

1. The language in the specification very carefully recommends the 
clock precision without requiring it.  It is expected to work over 
routers with jitter and delays and with media components from outside 
the broadcast facility. The difference is that the client is expected 
to be able to perform tight sync when the servers are able to provide 
the recommended timing model (encoding one hop away, etc).  It 
gracefully degrades to "Internet" timing model quality.

2. The clock, although "NTP" syntax, is not time of day (e.g. 
reference_ID="GPS"), but "PPS", thus allowing the components to be 
encoded and decoded without concern for end-to-end delivery 
delay.  This only works when NTP is explicitly signaled as part of 
the encoding and both components share a clock (same or closely 
coupled encoders). So it will not work in the general case.  But 
there are many systems where it will and perhaps consideration should 
be given to permitting non-"wall clock" timebases.

The NTP hdr ext here is an encoding optimization that indeed has some 
other good properties.  Although the draft ties the NTP timestamp 
back to the RTCP SR records, the timebase is not explicitly defined 
(which is also true of RTCP SR).  Some reference_ID value must be 
assumed and there is enough delta between, for example, GPS and UTC, 
timebases to cause things to break if the entire system is not 
implicitly assumed to be only one timebase. For example, without an 
explicit reference_ID, how could one use other timebases defined in 
NTPv4?  This is a broader issue than the hdr ext syntax, but since 
we're discussing NTP, I thought I'd bring it up.

I think the above and the other points below are relevant here and 
not inconsistent with the RTCP SR methods. The fact that the ATSC 
design *can* be more precise is not alone cause to discard the 
techniques developed there.

Regards,

         Mike

At 09:34 AM 3/31/2009 +0200, [email-masked] wrote:
>I guess, what we are looking for in AVT is a solution which also 
>works for encoders having non-perfect clocks. There was a long 
>discussion on the list, here are two pointers:
>
>ATSC surely can assume to have a perfect clock in the NTP streams, 
>which will not be the case on PC systems that may be used as an 
>encoder for multi session transmission of a layered codec. This is 
>one major problem we intended to solve with the header extension 
>while keeping the whole approach as close as possible in line with 
>the existing RTCP SR method.
>
>
>Best regards,
>Thomas
>
>
>
>>3. The above works in multicast-only, UDLR networks (e.g. 
>>television broadcast), rather than requiring a 2-way link.
>>
>>them arrive out of sync with the reference frames delays the 
>>service acquisition by half the period on average.
>>
>>arguably be more efficiently managed by the server in most cases, 
>>not individual client requests for faster RTCP SR packets. And, the 
>>frame periods being used by the server, so it would not, in 
>>general, know what to ask for.
>>