Subject: [AVT] How to correctly group multiple source of the same media type when using layered codecs

From: Magnus Westerlund <[email-masked]>
Date: Wed, 03 Dec 2008 19:29:00 +0100

Hi,

Another issue with layered codecs is how to determine which SSRC's in
the different layers actually are part of the same source. The immediate
reaction is of course to use SDES cname mechanism and group the SSRC
with the matching cname in each layer. That works as long as one doesn't
have more than one source (microphone, video camera, etc) in the same
RTP session that needs to be synchronized.

In the non-multilayered case this is not an issue, as each SSRC is
providing a completely contained stream and encoding. But in the
multilayered case you can get this:

Layer a:

SSRC X: CNAME=Charlie
SSRC Y: CNAME=Charlie

Layer b:

SSRC Z: CNAME=Charlie
SSRC Q: CNAME=Charlie

So how do you determine that X and Q are one source and Y and Z are the
other?

The immediate solution is of course align the SSRC values between the
layers. An alternative is to define additional SDES items that provides
a second layer of information beyond that they needs to be synchronized.

Other solutions or errors in my thinking?

Cheers

Magnus Westerlund
(With the individual contributor hat on)

IETF Transport Area Director & TSVWG Chair
----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Alex Eleftheriadis <[email-masked]>
Date: Thu, 4 Dec 2008 12:05:02 +0200

The current thinking, exemplified in the draft RTP payload format for  
SVC, is that the SSRC values between the layers are identical. In  
various discussions between the editors and others we could not  
identify any drawbacks to this. If someone can think of one, now would  
be a good time to say so as the draft is nearing completion.

--Alex

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 04 Dec 2008 11:24:53 +0100

Alex Eleftheriadis skrev:
> various discussions between the editors and others we could not identify
> any drawbacks to this. If someone can think of one, now would be a good
> time to say so as the draft is nearing completion.
> 

RFC 3550 does say that it is the recommended behavior and I would argue
that it is mandatory to use this solution due to the raised issue.

I haven't seen a issue with using the same SSRC over all the layers.
What I am a bit concerned is the RFC 3550 text in section 8.3:

   For layered encodings transmitted on separate RTP sessions (see
   Section 2.4), a single SSRC identifier space SHOULD be used across
   the sessions of all layers and the core (base) layer SHOULD be used
   for SSRC identifier allocation and collision resolution.  When a
   source discovers that it has collided, it transmits an RTCP BYE
   packet on only the base layer but changes the SSRC identifier to the
   new value in all layers.

The issue I have is with the SHOULD use the core/base layer for
collision detection. If one has a codec that lacks base layer or allow
other codecs or payload types that is differently configured to use the
same RTP sessions one may end up with the case where an SSRC is not
first used in the core/base layer RTP session. Thus I would recommend
that one has detection in all layers and in case of collison applies the
change in all.

Cheers

Magnus Westerlund

---

From: "Jonathan Lennox" <[email-masked]>
Date: Thu, 4 Dec 2008 11:59:44 -0500

I agree with your conclusion, that SSRC collision detection should be done on all the associated sessions, and SSRCs should be changed everywhere if they collide anywhere.  I think this would technically require a document updating 3550, but it's fairly minor.

The one concern I've had is if there could be codecs where it's possible for different session members to (usefully) participate in non-overlapping subsets of the sessions.  In this case, cross-layer SSRC collisions wouldn't be detected.  The question is whether this a) could happen, and b) would actually matter.  (For instance, I would argue if a member isn't sending media, but is only providing RTCP feedback, it doesn't.)

-- 
Jonathan Lennox
Vidyo, Inc
[email-masked]

> -----Original Message-----
> From: Magnus Westerlund [mailto:[email-masked]]
> Sent: Thursday, December 04, 2008 5:25 AM
> To: Alex Eleftheriadis
> Cc: IETF AVT WG
> Subject: Re: [AVT] How to correctly group multiple source of the same
> media type when using layered codecs
> 
> identify

---

From: <[email-masked]>
Date: Fri, 5 Dec 2008 10:52:34 +0200

>SSRC Q: CNAME=Charlie

Why would you use two SSRCs for one layer?

BR, YK 

>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of ext Magnus Westerlund
>Sent: 03 December, 2008 20:29
>To: IETF AVT WG
>Subject: [AVT] How to correctly group multiple source of the 
>same media type when using layered codecs
>
>Hi,
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 09:55:47 +0100

> Why would you use two SSRCs for one layer?

Please read my previous email again. There is two SSRC's because they
are two different media sources, i.e. two microphones or two cameras.
However, they are supposed to be synced because they are coming from the
same room and system.

Cheers

Magnus Westerlund

---

From: <[email-masked]>
Date: Fri, 5 Dec 2008 10:59:56 +0200

Actually we had some email discussions right before the past IETF meeting on this list, and as it is possible to that SVC may have indepenent subsets of layers, I proposed that each of those independent subsets MUST use the same SSRC, while it is allowed for the different independent subsets to use different SSRCs. Nobody expressed an objection. Therefore, I have included in the latest under-development SVC draft. 

Example: There are 5 layers, a, b, c, d, and e, where a is the base layer. Layer c depends on layer a, and layer e depends on layer c. However, layer b does not depend on any other layer, and layer d depends on layer b. Therefore, there are two independent subsets: a+c+e, and b+d. Layers a+c+e MUST use the same SSRC, and layers b+d MUST use the same SSRC. However, layers a and b may use different SSRCs. 

BR, YK 

>Behalf Of ext Alex Eleftheriadis
>Sent: 04 December, 2008 12:05
>To: Magnus Westerlund
>Cc: IETF AVT WG
>Subject: Re: [AVT] How to correctly group multiple source of 
>the same mediatype when using layered codecs
>
>On Dec 3, 2008, at 8:29 PM, Magnus Westerlund wrot
>
>> Hi,
>>

---

From: <[email-masked]>
Date: Fri, 5 Dec 2008 11:14:05 +0200

OK, this is like layered multiview video or layered multi-channel audio. Let's use multiview video as the example to ease the discussion. I suppose you meant that the two views are encoded in one video bitstream. But you don't have to use two SSRCs in transport to indicate that they are from different cameras, because in the video bitstream there are view/camera identifiers signaled, similarly as done in the MVC standard. For transport of such bitstreams, we should also allow even data from different views to be transported within one RTP packet - in that case you cannot use different SSRC anyway. 

BR, YK 

>From: ext Magnus Westerlund [mailto:[email-masked]] 
>Sent: 05 December, 2008 10:56
>To: Wang Ye-Kui (Nokia-NRC/Tampere)
>Cc: [email-masked]
>the same media type when using layered codecs
>

---

From: <[email-masked]>
Date: Fri, 5 Dec 2008 11:21:06 +0200

Here is my understanding to the RFC 3550 text excerpted below. It was assumed that all the layers depend on the base/core layer, and with this assumption the text should be OK, as it is said that the core layer is used for SSRC detection and resolution, but once a resolution is needed, it applies to all the layers. 

However, I am not sure whether the intention of the wording "a single SSRC identifier space" was "a single SSRC identifier value".

BR, YK 

>Behalf Of ext Jonathan Lennox
>Sent: 04 December, 2008 19:00
>To: Magnus Westerlund; Alex Eleftheriadis
>Cc: IETF AVT WG
>I agree with your conclusion, that SSRC collision detection 
>should be done on all the associated sessions, and SSRCs 
>should be changed everywhere if they collide anywhere.  I 
>think this would technically require a document updating 3550, 
>but it's fairly minor.
>
>The one concern I've had is if there could be codecs where 
>it's possible for different session members to (usefully) 
>participate in non-overlapping subsets of the sessions.  In 
>this case, cross-layer SSRC collisions wouldn't be detected.  
>The question is whether this a) could happen, and b) would 
>actually matter.  (For instance, I would argue if a member 
>isn't sending media, but is only providing RTCP feedback, it doesn't.)
>
>-- 
>Jonathan Lennox
>Vidyo, Inc
>[email-masked]
>
>

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 10:47:34 +0100

> OK, this is like layered multiview video or layered multi-channel audio. Let's use multiview video as the example to ease the discussion. I suppose you meant that the two views are encoded in one v…

No, I am not talking about any such fancy encoding. I am talking about
one camera goes to one encoder producing layered output being sent in
the same RTP session as another camera->encoder chain producing a
layered output. In this case there is one single node that produces both
streams and thinks these two shall be synchronized. It is a basic
functionality of RTP to support this usage. Each camera has its own SSRC
to represent it as unique sender. If the two cameras where not to be
synchronized they would have the different CNAMEs.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 10:52:01 +0100

> Here is my understanding to the RFC 3550 text excerpted below. It was assumed that all the layers depend on the base/core layer, and with this assumption the text should be OK, as it is said that t…

Yes, that is my interpretation also. And that is why I think there is a
problem with the text. With multi descriptive coding there is no base
layer. Thus one needs to modify this.

> However, I am not sure whether the intention of the wording "a single SSRC identifier space" was "a single SSRC identifier value".

No, it really are talking about the identifier space. The SSRC space is
all possible values, i.e. 2^32 which is the SSRC space. In this case the
talk about a single SSRC identifier space is saying that the SSRC shall
be equal in all layers/rtp sessions that are used together to represent
the different sources.

So this text do recommend SSRC alignment between the layers. As stated
before I don't see a problem with it and to avoid the problem that
started this thread I think it is a MUST rather than a SHOULD.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 10:56:31 +0100

> Example: There are 5 layers, a, b, c, d, and e, where a is the base layer. Layer c depends on layer a, and layer e depends on layer c. However, layer b does not depend on any other layer, and layer…

No, I think that is wrong. As the SSRC represent the actual source, i.e.
camera that captures the material that is provided the result of
allowing this is that a+c+e representation looks like a different camera
then the b+d one. Thus a receiver that sees all the layer will try to
render both as they appear to be different sources.

To me the right thing is to use the same SSRC value for all layers
produced from a specific video input.

Cheers

Magnus Westerlund

---

From: Colin Perkins <[email-masked]>
Date: Fri, 5 Dec 2008 10:19:13 +0000

I guess the main issue is that this prevents you from sending the  
different layers in the same RTP session (so-called "SSRC  
multiplexing").  Having the ability to do that, and to trivially  
gateway between one UDP port per layer and one UDP port for the  
entire stream always seemed a useful feature.

We might consider either signalling the dependencies, or using  
something like a new SDES item to indicate which stream correspond to  
which media source instead.

Colin

On 4 Dec 2008, at 10:05, Alex Eleftheriadis wrote:
> The current thinking, exemplified in the draft RTP payload format  
> for SVC, is that the SSRC values between the layers are identical.  
> In various discussions between the editors and others we could not  
> identify any drawbacks to this. If someone can think of one, now  
> would be a good time to say so as the draft is nearing completion.
>
> --Alex
>

-- 
Colin Perkins
http://csperkins.org/

---

From: Thomas Schierl <[email-masked]>
Date: Fri, 05 Dec 2008 12:14:13 +0100

> started this thread I think it is a MUST rather than a SHOULD.
The MUST + mandating "collision detection in all layers" is probably 
enough. The only corner case has been mentioned by Jonathan: if there is 
no overlapping between the sessions. I think that one can be ignored. 
Since a real collision would only happen, if the receivers collide in 
the same session. If they just collied inter session-wise, the source 
may get temporally a wrong picture on the feedback given by the 
receivers, as long as no CNAME bounding is given. So, this is really minor.

But in case of SSRC multiplexing, which has been explicitly excluded 
from the SVC draft, such an approach does not work. Then further 
extensions would be required.

-- 
Thomas Schierl
--------------
Fraunhofer HHI

---

From: <[email-masked]>
Date: Fri, 5 Dec 2008 13:39:01 +0200

If one builds such a system, he/she should associate the camera identifier/position information to each of the streams, to enable proper display. Simply using RTP and different SSRC values to differentiate different cameras seems not sufficient for such use cases anyway. 

BR, YK 

>Sent: 05 December, 2008 11:48
>> OK, this is like layered multiview video or layered 
>multi-channel audio. Let's use multiview video as the example 
>to ease the discussion. I suppose you meant that the two views 
>are encoded in one video bitstream. But you don't have to use 
>two SSRCs in transport to indicate that they are from 
>different cameras, because in the video bitstream there are 
>view/camera identifiers signaled, similarly as done in the MVC 
>standard. For transport of such bitstreams, we should also 
>allow even data from different views to be transported within 
>one RTP packet - in that case you cannot use different SSRC anyway. 
>

---

From: Colin Perkins <[email-masked]>
Date: Fri, 5 Dec 2008 11:52:09 +0000

On 5 Dec 2008, at 11:14, Thomas Schierl wrote:
>>> However, I am not sure whether the intention of the wording "a  
>>> single SSRC identifier space" was "a single SSRC identifier value".
>>
>> No, it really are talking about the identifier space. The SSRC  
>> space is all possible values, i.e. 2^32 which is the SSRC space.  
>> In this case the talk about a single SSRC identifier space is  
>> saying that the SSRC shall be equal in all layers/rtp sessions  
>> that are used together to represent the different sources.
>>
>> So this text do recommend SSRC alignment between the layers. As  
>> stated before I don't see a problem with it and to avoid the  
>> problem that started this thread I think it is a MUST rather than  
>> a SHOULD.
>>
> The MUST + mandating "collision detection in all layers" is  
> probably enough. The only corner case has been mentioned by  
> Jonathan: if there is no overlapping between the sessions. I think  
> that one can be ignored. Since a real collision would only happen,  
> if the receivers collide in the same session. If they just collied  
> inter session-wise, the source may get temporally a wrong picture  
> on the feedback given by the receivers, as long as no CNAME  
> bounding is given. So, this is really minor.
>
> But in case of SSRC multiplexing, which has been explicitly  
> excluded from the SVC draft, such an approach does not work. Then  
> further extensions would be required.

My gut reaction this that we're introducing special cases here, which  
will hurt us later.  Non-overlapping sessions and SSRC multiplexing  
for layered codecs are obvious generalisations, that I expect someone  
will want to use in the future.  It may be worth taking a little  
extra complexity now, to avoid a lot of extra complexity if we have  
to engineer those features in later.

-- 
Colin Perkins
http://csperkins.org/

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 13:16:52 +0100

> If one builds such a system, he/she should associate the camera identifier/position information to each of the streams, to enable proper display. Simply using RTP and different SSRC values to diffe…

It is still the a fundamental mechanism for RTP in non-multi-layered
codecs. Breaking that for a layered codec simply is wrong.

I know people doesn't like this in Video because of the display issue,
but for audio it has much less problems as mixing audio together or
doing spatial rendering to ease separation works quite well.

Cheers

Magnus Westerlund

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 13:23:31 +0100

Colin Perkins skrev:
> will hurt us later.  Non-overlapping sessions and SSRC multiplexing for
> layered codecs are obvious generalisations, that I expect someone will
> want to use in the future.  It may be worth taking a little extra
> complexity now, to avoid a lot of extra complexity if we have to
> engineer those features in later.
> 

Colin, I agree that this is making an restriction for the current usage.
But is an restriction that already is present in RFC 3550. We will need
to have additional mechanism to support SSRC multiplexing. So are there
an issue of mandating SSRC alignment for now and then develop the
solution for SSRC multiplexing later and require support of the defined
extensions at that point? I personally don't see how the solution gets
more complex later.

The most obvious solution seems to add an addtional SDES item that binds
SSRCs together as being parts of one media source. But as it is so
simple we could define it now just to make it clear that the alternative
exist when needing solutions in that space.

Cheers

Magnus Westerlund

---

From: Colin Perkins <[email-masked]>
Date: Fri, 5 Dec 2008 13:55:08 +0000

> extensions at that point? I personally don't see how the solution gets

The SSRC alignment for layered codec part of RFC 3550 only works if  
there's a common base layer.  My feeling is that that won't be the  
case for long, so we should deprecate that part of 3550, and work on  
a more general solution.

> exist when needing solutions in that space.

Agree that a new SDES item is likely required.

-- 
Colin Perkins
http://csperkins.org/

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 05 Dec 2008 15:25:46 +0100

Colin Perkins skrev:
> there's a common base layer.  My feeling is that that won't be the case
> for long, so we should deprecate that part of 3550, and work on a more
> general solution.

Agreed, we need a document that collects all these general layered codec
 issues and recommended solutions to them including some updates to RFC
3550.

Cheers

Magnus Westerlund

---

From: Thomas Schierl <[email-masked]>
Date: Fri, 05 Dec 2008 15:35:20 +0100

>  issues and recommended solutions to them including some updates to RFC
We already have a document which identified most of the layered codec 
issues, but may need to be extended with respect to solutions:

http://tools.ietf.org/html/draft-schierl-avt-rtp-multi-session-transmission-00

-- 
Thomas Schierl
--------------
Fraunhofer HHI