Subject: Re: IETF AVT last call on draft-ietf-avt-rtp-mpeg4-es-03.txt

From: Colin Perkins <[email-masked]>
Date: Tue, 29 Aug 2000 15:47:04 +0100

--> Leonid Rosenboim writes:
>
>Colin Perkins wrote:
>
>> --> Leonid Rosenboim writes:
>> >from a quick look at the latest draft, I have not seen any references to the issue of
>> >combined Audio/Video streaming. I understand that with this scheme Video and Audio
>> >will each have its own RTP session, which are basically independent of one another,
>> >except the initiation which is done with SDP or H.245.
>> >
>> >Still, perhaps there should be some specific description of how these streams should
>> >be related, for instance to make sure that the Audio and Video do not run out of sync
>> >with respect to one another. Should in such a case both Audio and Video timestamps
>> >begin with the same random value and share the same timestamp resolution ? Surely,
>> >they should at least have a common denominator or something.
>>
>> The synchronisation is achieved using standard RTCP sender report packets
>> and the RTP timestamp.
>>
>
>Perhaps it would be simplier if the payload would specify that in the
>audio+video case, both audio and video sessions should have a common
>timestamp initial value and resolution, as well as a common SSRC. This
>draft is all about simplicity, so why make it more complex by mandating
>RTCP sender report?

Because this draft is supposed to interwork with existing RTP implementations.

>Consider a situation where a few video packets got lost right at the
>beginning of a session, but none of the audio packets got lost, the player
>will have to assume the initial timestamp for the video is of the first
>packet arrived, hence all subsequent video will loose sync with the audio,
>as the audio initial timestamp will be correct, but vide initial timestamp
>will not be correct.

Not true. The periodic sender reports allow you to establish the mapping. 

>> >Also, since each will probably have a different payload type, they
>> >could share the same port numbers, same SSRC, but could for example
>> >have difference IP Precedence values if during network congestion it is
>> >desired to still have Audio when the Video can not make it through.
>>
>> Yes, that is a possibility.
>
>I suggest this is "recommended" in the draft.

No, this is outside the scope of a particular payload format. There are
many ways of protecting media streams, the payload should work with all
of them, rather than mandating a particular solution.

Colin

---

From: Colin Perkins <[email-masked]>
Date: Wed, 30 Aug 2000 08:38:48 +0100

--> Leonid Rosenboim writes:
>> [snip]
>Do you mean existing implementations of this draft ? I do not see how
>my suggestion hurts implementations of other payload types, as the tie
>between Audio and Video timing I suggested was specific for this payload
>type only.

No, I mean with other payload types. For example using MPEG-4 video with
non-MPEG-4 audio, and wishing to synchronize them.

>In that case, RTCP sender reports and support of NTP time reference
>needs to be MANDATED by this payload type. Aren't sender reports
>optional ?

Not if you need synchronization.

>Do you really expect mobile phones to have NTP implementations embedded
>in them ?

No. I expect them to be able to understand NTP format timestamps. There is
no requirement in RTP for the clock to be NTP-synchronized (so long as the
sender is consistent, the clock can have an arbitrary reference).

>Besides, I can see only one justification for having two different payload formats
>for MPEG-4: one needs to be simple to implement, which is what this one is suppoed
>to do, hence my suggestion to make it simplier.

You are simplifying this format, at the expense of making interworking with
other RTP implementations harder. The reason for developing this format is
to work with H.323 terminals, which use standard RTP implementations.

Colin

---

From: CURET Dominique FTRD/DMI/REN <[email-masked]>
Date: Wed, 30 Aug 2000 10:08:11 +0200

Dear all,

because of delays due to different buffer sizes, to average 
bitrates of elementary streams, the time needed for bytes to get 
across audio and video buffers will be completely different.
If you want to save buffers, when Video and audio are multiplexed 
together the audio shouild be tranmitted quite late. That is to 
say that when you see some video passing through a pipe,
you have to wait quite a while to see the audio that will
be reproduced in phase.

regards

_________________________________________
> Dominique Curet
> France Télécom R&D - DMI/PIA
> % + 33 (0)2 99 12 40 05   Fax : + 33 (0)2 99 12 40 98
> )  CCETT       B.P. 59            4, rue du Clos Courtel 
> 35512           Cesson-Sevigne   Cedex   09     FRANCE
> mailto:[email-masked]
_________________________________________

-----Message d'origine-----
De: Colin Perkins [mailto:[email-masked]]
Date: mercredi 30 août 2000 09:39
À: [email-masked]
Cc: [email-masked]; [email-masked]
Objet: Re: IETF AVT last call on draft-ietf-avt-rtp-mpeg4-es-03.txt 

Colin

---

From: Stephan Wenger <[email-masked]>
Date: Wed, 30 Aug 2000 18:24:03 +0200

>you have to wait quite a while to see the audio that will

Dominique,

Although your statement is true from a theoretical point-of-view,
it is not what most H.323 systems do these days.  An H.323
terminal sends media data as soon as the coding/packetization
process is finished.  The lip-synchronization is performed at
the receiver by delaying reproduction of audio (or, in rare cases,
video) until synchronization is achieved.  Many systems have
some settings in the user interface that allow not to use lip-
synchronization, in which case reproduction starts as soon as
the RTP buffering mechanism delivers the packets.

All this is, of course, due to the significantly higher algorithmic
delay of the video codec compared to the audio codec.  The
argument that the packetization delay for audio is high (because
it takes time at low audio bitrates to fill a packet) may, again,
be true from a theoretical point-of-view.  Real-world systems,
however, use fairly small packets and accept the bad payload/
overhead ratio to deliver audio as soon as practical.

Finally, sender based media synchronization does not work in
multicast scenarios, because there may be different routings
for streams going to the SAME receiver, leading in different
time offsets between streams to the SAME receiver, yielding
different time offsets between DIFFERENT receivers.

Stephan