Subject: Re: Ability To Synchronize Streams

From: Tom-PT Taylor <[email-masked]>
Date: Wed, 3 Nov 1999 08:13:28 -0500

Streams on different terminations are correlated by StreamID: StreamID is
context-wide.  In the case you describe where there is video on one
termination and audio on the other, there is no synchronization to be done
-- neither termination handles both streams.  Or maybe you mean to say that
one termination is receiveOnly for audio while the other termination is
receiveOnly for video.  Same thing -- no need for synchronization until at
least one termination changes to a mode where it is sending out both
streams.  At that point, the rule that says that all streams in a context
are synchronized to the extent this is meaningful assures the required
synchronization.

> -----Original Message-----
> From: John Segers [SMTP:[email-masked]]
> Sent: Monday, November 01, 1999 8:17 AM
> To:   [email-masked]
> Subject:      Re: Ability To Synchronize Streams
>
> Actually, there was an oversight.  In the model that came out of
> Santiago, all streams coming from one user went into the same
> Termination.  The streams within one Termination were synchronised by
> default.  In the discussion following the Santiago meeting, we changed
> the model for multimedia.  We do not have a mechanism to specify that
> streams on different Terminations have to be synchronised.
>
> Specifically, if there is an RTP video stream on one Termination and an
> RTP audio stream on another, the protocol currently cannot specify that
> they have to be synchronised.  We can either add a synchronisation
> mechanism or mandate that streams that have to be synchronised must
> "pass through" one Termination.
>
> I would say that a synchronisation mechanism is best done at the context
> level because it involves multiple Terminations and because I can't
> think of a case where you would want to streams to be synchronised for
> one user, but not for the other.
>
> John
>
> > Tom-PT Taylor wrote:
> >
> > The thought that streams are all synchronized by default may be the
> > answer.  I confess I had forgotten.  This is something the multimedia
> > specialists will have to think about.
> >
> >      -----Original Message-----
> >      From:   Chip Sharp [SMTP:[email-masked]]
> >      Sent:   Thursday, October 28, 1999 11:41 AM
> >      To:     Taylor, Tom-PT [CAR:B904-I:EXCH];
> >      [email-masked]
> >      Subject:        Re: Ability To Synchronize Streams
> >
> >      It would definitely be easier to put it off.
> >
> >      One of the main reasons streams were introduced were for
> >      multimedia which
> >      requires at least lip-synch.
> >
> >      My memory of Santiago was that streams in a context were synched
> >      by
> >      definition.  I thought that was the main reason for streams.
> >      Otherwise, we
> >      could get rid of streams and just have each media stream in its
> >      own context
> >      with a (synch=ContextID) parameter indicating the contexts that
> >      should be
> >      synchronized.
> >
> >      Chip
> >
> >      At 10:09 AM 10/28/99 -0400, Tom-PT Taylor wrote:
> >      >This is to remind you of an oversight arising out of our
> >      multimedia
> >      >discussions last spring.  We need a way to indicate which
> >      streams are to be
> >      >synchronized (e.g. synchronization of voice with video) on exit
> >      from the MG.
> >      >My proposal was that such synchronization would be indicated at
> >      each
> >      >termination by including a "synch=StreamId" property within one
> >      stream
> >      >descriptor to point at another stream to which it should be
> >      bound.  On
> >      >second thought, it may be that we actually want to do this at
> >      the context
> >      >level, which means that the whole thing should be deferred for
> >      version 2
> >      >like the bridging context property has been.
> >      >
> >      >Comments?
> >      >
> >      >Tom Taylor
> >      >Advisor -- IPConnect Standards
> >      >E-mail: [email-masked] (internally, Tom-PT Taylor)
> >      >Phone and FAX: +1 613 736 0961
> >
> >      Support NetAid!  http://www.netaid.org
> >      --------------------------------------------------
> >      Chip Sharp                 Consulting Engineering
> >      Cisco Systems              Telco Bio-region
> >      Reality - Love it or Leave it.
> >      --------------------------------------------------
>
> --
> John Segers                                  email: [email-masked]
> Lucent Technologies                                        Room HE 306
> Dept. Forward Looking Work                      phone: +31 35 687 4724
> P.O. Box 18, 1270 AA  Huizen                      fax: +31 35 687 5954

---

From: John Segers <[email-masked]>
Date: Wed, 3 Nov 1999 14:48:35 +0100

Tom,

What you say is correct, but does not address the problem I referred to.

Consider an audio/video call.  On the IP side, there would be two
Terminations for the one call: one for audio and one for video.
Synchronization of the streams on these two Terminations could be
desired (lip synch).  I don't see how a rule like the one you mention
would achieve this.

John

> Tom-PT Taylor wrote:
>
> Streams on different terminations are correlated by StreamID: StreamID
> is context-wide.  In the case you describe where there is video on one
> termination and audio on the other, there is no synchronization to be
> done -- neither termination handles both streams.  Or maybe you mean
> to say that one termination is receiveOnly for audio while the other
> termination is receiveOnly for video.  Same thing -- no need for
> synchronization until at least one termination changes to a mode where
> it is sending out both streams.  At that point, the rule that says
> that all streams in a context are synchronized to the extent this is
> meaningful assures the required synchronization.
>

---

From: Chip Sharp <[email-masked]>
Date: Mon, 22 Nov 1999 17:09:26 -0500

I agree.  In the RTP case, most likely both streams will go to the same IP
address with a different port.

I had assumed out of Santiago that streams in a context were synchronized
by default.  Since contexts are an artificial construct anyway this
shouldn't be too hard.

Are you suggesting that we have a synchronization parameter for contexts
similar to the topology parameter?

BTW,  SDP should be able to provide a session description for both audio
and video from the same RTP termination.  That would be one way of doing it
for the SDP encoding.  I don't know about the binary version.

Chip

---

From: John Segers <[email-masked]>
Date: Tue, 23 Nov 1999 09:48:29 +0100

Chip,

Apparently you have a different understanding of the Santiago model than
I do.  At the Santiago meeting Tom suggested that a Termination would
represent a user (or at least the streams to/from a user for one
session).  The idea was that synchronization would be specified among
streams (voice synchronized with video, but not with data).  I don't
remember that we resolved the question whether it would be done at the
Context level or the Termination level.

The discussions after Santiago led to a rejection of this notion that a
Termination contains the streams to/from one user.  My understanding was
that we have to at least allow multiple streams to be on different
Terminations.  It seems that we don't have to allow multiple,
synchronized streams to/from one user on different Terminations (except
when they are multiplexed on to several physical Terminations).

Nevertheless, if we want to consider multimedia with audio, video and
data, we cannot simply assume all streams in the Context to be
synchronized.  If I have a session with three streams (audio,
video-speaker, video-document-camera), I don't necessarily lipsync all
three.  I think we need to be able to specify which streams to
synchronize.

Given the time frame and the more important matters (events, signals!),
I propose that we assume for now that 1) streams to be synchronized will
be on one Termination, and 2) all streams on the same Termination are
synchronized.  (I have to admit that I don't know if 2) makes sense if a
termination uses different types of transport for different streams).

John

---

From: Christian Groves <[email-masked]>
Date: Tue, 23 Nov 1999 13:37:51 +0100

G'Day all,

While I definately think a full blown solution for synchronisation and
multimedia is needed in CS2 I think some text similar to what John has
mentioned below may be beneficial for basic multimedia services to be
supported in this version.

I think its important to mention that 'outgoing streams' are
synchronised.

I don't see that all streams on a termination should be synchronised ie.
Audio and Video should be for lipsync but not a data application ie.
whiteboarding. All these may be on the once termination.

Cheers, Christian

---

From: Chip Sharp <[email-masked]>
Date: Tue, 23 Nov 1999 14:24:57 -0500

>Context level or the Termination level.

It is very likely we have different understandings of the Santiago
model.  However, I do remember what you say above.  Streams were brought
into being along with the Mux termination.  Other than a mux termination, I
don't remember much reason for streams.

>when they are multiplexed on to several physical Terminations).

I believe the only real application of streams was for Mux terminations,
specifically H.320 mux terminations.  In this case, the streams represent
the various media streams within the Mux.  For H.320, this is voice, video
and data (possibly control).

>termination uses different types of transport for different streams).

This is probably the best approach we can hope for for right now.  If it
doesn't make sense to synchronize streams on the same termination, we'll
have to assume the MG will do the right thing.  A T.120 data stream doesn't
necessarily have to be synchronized with video, but a text conversation
stream might.

This should be added to the text.  Until someone comes up with an
application for which this doesn't work or volunteers to address in detail
how a H.320 gateway/MCU will really work, this will have to do.

Chip

---

From: Danny Levin <[email-masked]>
Date: Wed, 24 Nov 1999 09:05:28 -0500

> similar to the topology parameter?

I think that this is a good idea to put routing, association and
synchronization of streams in the Topology descriptor
(or add a similar descriptor).
(RTP is a private case and in addition it does not always help.)

If we are not ready now (for first release) we can at least say that this is
a place holder for upcoming releases .

Danny.

> From: Media Gateway Control (megaco)
> [mailto:[email-masked]]On Behalf Of Chip Sharp
> Sent: Monday, November 22, 1999 5:09 PM
> To: [email-masked]