Subject: Re: [clue] Do we need the encodings as secified in the framework document - proposal to remove them from the framework

From: Paul Kyzivat <[email-masked]>
Date: Thu, 01 Nov 2012 12:38:49 -0400

(As co-chair)

Roni is proposing a significant change here.

PLEASE carefully consider this and comment on it here on the mailing list!

*If* this works and satisfies all objectives then it will be a helpful 
simplification. But it may be that simplifying things this way excludes 
some required behavior.

This is something that would benefit from discussion next week. But that 
won't be helpful if people haven't already thought about it.

	Thanks,
	Paul

On 10/31/12 5:49 AM, Roni Even wrote:
> Hi,
>
>  From the email discussion it looks to me that the concept of the
> encoding as specified in the framework document is not clear. I will try
> to describe my understanding of sections 7 and 8 of the framework
> document and explain why I think that it is not important to CLUE as
> specified. I am looking for response if my understanding of the encoding
> based on the framework is correct and how it will work for the examples
> I will provide bellow
>
> The purpose of the Encodings is to provide INFORMATION about the
> provider abilities to send streams. This relates to the available
> resources (compute and BW) of the providers.
>
> The encodings are constructed from encoding groups and each encoding
> group is include individual encodes.
>
> The encoding group provide a value for the total maximum BW and compute
> H264Mbps (note that it is H.264 specific) of all the individual encodes
> in the group that the provider can use to send audio and video.
>
> Each encoding group include a list of individual encodes  (audio and /or
> video). These provide maximum values that can be used to instantiate a
> media stream by the provider.
>
> Each video individual encodes identified by an encodeID provides maximum
> values  for BW, compute, resolution and frame rate for H.264. (BTW: the
> maxH264Mbps can be computed from the resolution and frame rate as
> defined in table 2 making it redundant).
>
> Each audio encode has only a BW value (no identifier).
>
> A media capture is associated with an encoding group !!!!! not an
> individual encoding.
>
> The provider can use one of the individual encoding to instantiate a
> media capture, each individual encoding can be used by one media
> capture, the decision of which one to use according to the framework is
> by the provider. The consumer does not select an individual encode.
>
> The maximum number of streams that can result from a particular encoding
> group is equal to the number of individual encodings in the group (note
> that this is why the example in the current data model allows only for
> one video and one audio streams).
>
> Section 8 of the framework talk about having more than one individual
> encoding assigned to a media capture (simulcast) and also claims that it
> can be done by the consumer but there is no way to do it since the media
> capture is associated only with an encoding group. There is no way for a
> consumer even to assign a specific individual encoding to a group.
>
> My view is that this mechanism does not work and can be removed. The
> reasons are given bellow
>
> The encoding as currently specified is provided as information since
> there is no way to map one or more individual encoding to a specific
> media capture
>
> The video is H.264 specific and not general
>
> The relation between the individual encode and encoding group makes it
> difficult to achieve a reasonable set even if the consumer will be able
> to select the mapping. This is due to the fact the total defined by the
> group limits the usage for individual encoding. For example if  we have
> a three camera system and we set a value in the  group for
> maxGroupH264Mbps and want to allow for simulcast (two resolutions), we
> will need to define six individual encodes, three with high resolution (
> also high maxH264Mbps) and three lower resolution with lower
> maxH264Mbps. If the compute resource can do all six it will  reflected
> in the value of maxGroupH264Mbps but I assume that there is a constrain
> (otherwise why have encodings) making it lower. So if the consumer will
> ask for all six what will happen. Since there is more than one degree of
> freedom he will not be able to predict if he will get all or part on
> lower frame rate , if he will get only a subset of the streams, if he
> will get different resolutions.
>
> On the other hand if the 6 individual encodes will include lower values
> to allow for the group limit it will mean that a consumer not doing
> simulcast is limited by the fact the simulcast and ask (if possible) for
> only three streams, the individual encodes limit will be low so he will
> not be able to use the full compute resources.
>
> Section 8 concludes that the number of individual encodes in a group
> must allow for all media capture in a capture scene entry to be used
> simultaneously.
>
> So it look like this system does not work well for simulcast. What about
> no simulcast, is it needed. For example a three camera system that also
> have one presentation. My view that the cameras will be one capture
> scene and the presentation will be a second one. The question is if they
> all belong to one encoding group, here my view is “no” since they have
> different priority and should not compete for compute resources. So we
> will have two encoding groups one for presentation and one for the three
> cameras. For the three cameras each sending one stream I assume that the
> resource allocation will be a third for each so we will have three
> individual encode with the same values, this is the basic one and if it
> is a third than no need for advertising or configuring. Now if we also
> want to have an individual encode that has higher compute if the
> consumer wants only one capture (of the whole room), the group will have
> fourth one with higher value. So if there is way for consumer to
> configure individual encode when using three media captures, he may
> select the higher capability one with two lower ones. The total will be
> limited by the encoding group value and again it is not clear how the
> actual streams will be encoded to address the group limit since there
> are multiple options.
>
> To summarize I do not see value in the encoding as currently specified
> since they only provide information and no option for configuration. The
> value of it as information does not justify specifying them and if the
> intention is to allow configurations of individual encoding it must be
> reflected and the explained how it works.
>
> My proposal is at the moment to remove the encoding until we get some
> text that defines how to use it for configuring encoding to media
> captures including for the simulcast case
>
> Thanks
>
> Roni Even
>
>
>
> _______________________________________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/clue
>

---

From: Andy Pepperell <[email-masked]>
Date: Fri, 2 Nov 2012 15:20:18 +0000

>Paul:

I certainly agree with Paul's point here... however, with not all that many
of Roni's (as might be expected :) ) - to be specific:

>Roni:

video). These provide maximum values that can be used to instantiate a
media stream by the provider.****

Yes, I think that's a good summary.

** **

values  for BW, compute, resolution and frame rate for H.264. (BTW: the
maxH264Mbps can be computed from the resolution and frame rate as defined
in table 2 making it redundant).****

>Each audio encode has only a BW value (no identifier).****

Also agreed - apart from the H.264 point I think - a maximum resolution and
a maximum frame rate doesn't necessarily imply a maxMbps - many systems can
encode 1280x720 at some frame rate < 30 *or* some lower resolution at up to
30fps without necessarily being able to send 720p30, surely?

** **

** **

> Roni:

>A media capture is associated with an encoding group !!!!! not an
individual encoding.****

That's true, and by design. To re-iterate the original design intent here,
the idea behind the encoding groups idea was to be able to model 2 common
Telepresence architectures we see today:

1) a multi-screen endpoint room comprised of multiple endpoints (most
vendors' "3 screen" systems, for instance, tend to be composed of 3 of
their "1 screen" products linked together)

2) a flexible media generation system such as a software or hardware MCU

For the "1)" case above, due to real-world constraints such as camera
wiring etc., it seemed to us that a media capture representing, say, the
left camera of 3, would only be able to be encoded by the hardware it was
directly connected to - thus, such a system would use one "encoding group"
for its left constituent, one for its centre and one for its right.

>Roni:

>The provider can use one of the individual encoding to instantiate a media
capture, each individual encoding can be used by one media capture, the
decision of which one to use according to the framework is by the provider.
The consumer does not select an individual encode.****

That was not the intent - it should be the consumer that chooses media
capture / individual encoding pairs to instantiate "capture encodings".
When forming its configure message, the consumer essentially supplies a set
of media capture ID + encoding ID pairs to instruct the provider on which
capture encodings to send. Having just re-read it, I think that the
description as per Section 9 of the current framework document described
this well (though Mark deserves the credit for that, obviously!).

**>Roni: **

group is equal to the number of individual encodings in the group (note
that this is why the example in the current data model allows only for one
video and one audio streams).****

It is true that the number of streams that can result from a particular
encoding group is equal to the number of individual encodings in the group
- however, a fully flexible system might choose to have a single encoding
group and so no real restriction need be introduced by this (unless the
provider system has such restrictions - in which case they can be modelled
in this way).

>Roni:

encoding assigned to a media capture (simulcast) and also claims that it
can be done by the consumer but there is no way to do it since the media
capture is associated only with an encoding group. There is no way for a
consumer even to assign a specific individual encoding to a group.****

You're right that there's no way for the consumer to assign a specific
individual encoding to a group, but it's not clear what the need is - if a
provider system is sufficiently flexible that any encoding can be used by
any media capture, then it should construct its advertisement to use just a
single encoding group to comprise all encodings, at which point the
consumer has greater flexibility in which it can choose to be sent to it.

**>Roni: **

>The video is H.264 specific and not general

The problem we had was that we needed to come up with a way of describing a
big block of video encoding capability and how it could be subdivided. e.g.
you might be able to send 2 x 720p30 but not a single 720p60. The best
scheme we could come up with was to, for each of H.264, H.265 etc. include
some specific parameters that would allow both a provider's overall
capability to be signalled and how it might be subdivided across multiple
encodings. It's true that currently the parameters only really work for
H.264 and, moreover, only really work for a provider sending all encodings
within a group using the same codec, but the expectation was that new
codecs would have their own equivalent parameters added here in a similar
way.

**

>Roni:

>The relation between the individual encode and encoding group makes it
difficult to achieve a reasonable set even if the consumer will be able to
select the mapping. This is due to the fact the total defined by the group
limits the usage for individual encoding. For example if  we have a three
camera system and we set a value in the  group for maxGroupH264Mbps and
want to allow for simulcast (two resolutions), we will need to define six
individual encodes, three with high resolution ( also high maxH264Mbps) and
three lower resolution with lower maxH264Mbps. If the compute resource can
do all six it will  reflected in the value of maxGroupH264Mbps but I assume
that there is a constrain (otherwise why have encodings) making it lower.
So if the consumer will ask for all six what will happen. Since there is
more than one degree of freedom he will not be able to predict if he will
get all or part on lower frame rate , if he will get only a subset of the
streams, if he will get different resolutions.****

The situation you describe is, I would argue, exactly what the parameters
are for. The consumer would be allowed to ask for all 6, each with
consumer-imposed maximum values. The provider would be bound by its overall
maxGroupH264Mbps as well as each individual encoding's max. value - in many
senses this is no difference to existing systems' balancing of bit rate
across main and presentation limits. I think if the consumer asks for all
six then what will happen should be very predictable - 6 capture encodings
will be sent to the consumer, with each capped by the lower of the
consumer's request parameters and the provider's limitations. It is true
that the frame rate / resolution would be bounded but not fixed, but this
is true today on a video call - if you can receive 720p30 you don't know
whether the far end will send you 720p30, 720p5, CIF30 etc. I'm not sure I
see how what we're talking about here does anything but follow normal
encoder and decoder behavior as seen today (equally, I confess that I may
not be fully understanding the intricacies of the case you describe).

>On the other hand if the 6 individual encodes will include lower values to
allow for the group limit it will mean that a consumer not doing simulcast
is limited by the fact the simulcast and ask (if possible) for only three
streams, the individual encodes limit will be low so he will not be able to
use the full compute resources.

**

**

>Roni:

>Section 8 concludes that the number of individual encodes in a group must
allow for all media capture in a capture scene entry to be used
simultaneously.****

Yes, this is a corollary of the constraint that all media captures in a
capture scene entry must be able to be provided simultaneously (which
itself is a corollary of the idea that each capture scene entry is in
itself a useful representation of the scene).

** **

no simulcast, is it needed. For example a three camera system that also
have one presentation. My view that the cameras will be one capture scene
and the presentation will be a second one. The question is if they all
belong to one encoding group, here my view is “no” since they have
different priority and should not compete for compute resources.

Another view is that if the provider does encode the cameras and the
presentation from the same pool of encoding resource then it is valid for
the consumer to choose which to prioritise (i.e. the provider could leave
the consumer to allocate encodings to the media captures).

>Roni:

>So we will have two encoding groups one for presentation and one for the
three cameras. For the three cameras each sending one stream I assume that
the resource allocation will be a third for each so we will have three
individual encode with the same values, this is the basic one and if it is
a third than no need for advertising or configuring. Now if we also want to
have an individual encode that has higher compute if the consumer wants
only one capture (of the whole room), the group will have fourth one with
higher value. So if there is way for consumer to configure individual
encode when using three media captures, he may select the higher capability
one with two lower ones. The total will be limited by the encoding group
value and again it is not clear how the actual streams will be encoded to
address the group limit since there are multiple options.****

To my mind this is no different to the sort of trace offs / choices that
are already made in this sort of system today - in a video call it is
common to not be able to send a camera stream at full resolution and full
frame rate either because of bandwidth limitations or the capabilities of
the receiver. In the case you cite, yes, the consumer could choose to
impose different video limits on the left center and right cameras, but it
would be within its rights to do so, just as the provider would then be
within its rights to send them all at the lowest of the 3 levels.

**>Roni: **

since they only provide information and no option for configuration. The
value of it as information does not justify specifying them and if the
intention is to allow configurations of individual encoding it must be
reflected and the explained how it works.****

**>Roni: **

>My proposal is at the moment to remove the encoding until we get some text
that defines how to use it for configuring encoding to media captures
including for the simulcast case

I think I'd agree with you if it really were true that simulcast was
ill-defined here, but I think it's covered pretty well. It's clear that
you've thought about it in a lot of detail though, so I'm sure there'll be
more to discuss on this...

Regards,

Andy

On Thu, Nov 1, 2012 at 4:38 PM, Paul Kyzivat <[email-masked]> wrote:

>> ______________________________**_________________
>> clue mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/**listinfo/clue<https://www.ietf.org/mailman/listinfo/clue>
>>
>>
> ______________________________**_________________
> clue mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/clue<https://www.ietf.org/mailman/listinfo/clue>
>

---

From: "Roni Even" <[email-masked]>
Date: Mon, 5 Nov 2012 08:55:45 -0500

Andy,

Can you explain how the encoding works with the dynamic mapping of RTP to
Media Captures. 

Does this mean that the consumer will get advertisements from all the
participants of the multipoint call and will need to calculate the
limitations in each switch and ask for a new configuration since the group
limits may enforce different limitations.

In the static mapping case all this can be achieved using imageattr and the
codec parameters

Roni

From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy
Pepperell
Sent: 02 November, 2012 11:20 AM
To: Paul Kyzivat
Cc: [email-masked]
Subject: Re: [clue] Do we need the encodings as secified in the framework
document - proposal to remove them from the framework

>*If* this works and satisfies all objectives then it will be a helpful
simplification. But it may be that simplifying things this way excludes some
required behavior.

>Each encoding group include a list of individual encodes  (audio and /or
video). These provide maximum values that can be used to instantiate a media
stream by the provider.

values  for BW, compute, resolution and frame rate for H.264. (BTW: the
maxH264Mbps can be computed from the resolution and frame rate as defined in
table 2 making it redundant).

 

individual encoding.

1) a multi-screen endpoint room comprised of multiple endpoints (most
vendors' "3 screen" systems, for instance, tend to be composed of 3 of their
"1 screen" products linked together)

For the "1)" case above, due to real-world constraints such as camera wiring
etc., it seemed to us that a media capture representing, say, the left
camera of 3, would only be able to be encoded by the hardware it was
directly connected to - thus, such a system would use one "encoding group"
for its left constituent, one for its centre and one for its right.

capture, each individual encoding can be used by one media capture, the
decision of which one to use according to the framework is by the provider.
The consumer does not select an individual encode.

That was not the intent - it should be the consumer that chooses media
capture / individual encoding pairs to instantiate "capture encodings". When
forming its configure message, the consumer essentially supplies a set of
media capture ID + encoding ID pairs to instruct the provider on which
capture encodings to send. Having just re-read it, I think that the
description as per Section 9 of the current framework document described
this well (though Mark deserves the credit for that, obviously!).

>The maximum number of streams that can result from a particular encoding
group is equal to the number of individual encodings in the group (note that
this is why the example in the current data model allows only for one video
and one audio streams).

It is true that the number of streams that can result from a particular
encoding group is equal to the number of individual encodings in the group -
however, a fully flexible system might choose to have a single encoding
group and so no real restriction need be introduced by this (unless the
provider system has such restrictions - in which case they can be modelled
in this way).

>Section 8 of the framework talk about having more than one individual
encoding assigned to a media capture (simulcast) and also claims that it can
be done by the consumer but there is no way to do it since the media capture
is associated only with an encoding group. There is no way for a consumer
even to assign a specific individual encoding to a group.

You're right that there's no way for the consumer to assign a specific
individual encoding to a group, but it's not clear what the need is - if a
provider system is sufficiently flexible that any encoding can be used by
any media capture, then it should construct its advertisement to use just a
single encoding group to comprise all encodings, at which point the consumer
has greater flexibility in which it can choose to be sent to it.

within a group using the same codec, but the expectation was that new codecs
would have their own equivalent parameters added here in a similar way.

difficult to achieve a reasonable set even if the consumer will be able to
select the mapping. This is due to the fact the total defined by the group
limits the usage for individual encoding. For example if  we have a three
camera system and we set a value in the  group for maxGroupH264Mbps and want
to allow for simulcast (two resolutions), we will need to define six
individual encodes, three with high resolution ( also high maxH264Mbps) and
three lower resolution with lower maxH264Mbps. If the compute resource can
do all six it will  reflected in the value of maxGroupH264Mbps but I assume
that there is a constrain (otherwise why have encodings) making it lower. So
if the consumer will ask for all six what will happen. Since there is more
than one degree of freedom he will not be able to predict if he will get all
or part on lower frame rate , if he will get only a subset of the streams,
if he will get different resolutions.

that the frame rate / resolution would be bounded but not fixed, but this is
true today on a video call - if you can receive 720p30 you don't know

Yes, this is a corollary of the constraint that all media captures in a
capture scene entry must be able to be provided simultaneously (which itself
is a corollary of the idea that each capture scene entry is in itself a
useful representation of the scene).

>So it look like this system does not work well for simulcast. What about no
simulcast, is it needed. For example a three camera system that also have
one presentation. My view that the cameras will be one capture scene and the
presentation will be a second one. The question is if they all belong to one
encoding group, here my view is "no" since they have different priority and
should not compete for compute resources.

three cameras. For the three cameras each sending one stream I assume that
the resource allocation will be a third for each so we will have three
individual encode with the same values, this is the basic one and if it is a
third than no need for advertising or configuring. Now if we also want to
have an individual encode that has higher compute if the consumer wants only
one capture (of the whole room), the group will have fourth one with higher
value. So if there is way for consumer to configure individual encode when
using three media captures, he may select the higher capability one with two
lower ones. The total will be limited by the encoding group value and again
it is not clear how the actual streams will be encoded to address the group
limit since there are multiple options.

To my mind this is no different to the sort of trace offs / choices that are
already made in this sort of system today - in a video call it is common to
not be able to send a camera stream at full resolution and full frame rate
either because of bandwidth limitations or the capabilities of the receiver.
In the case you cite, yes, the consumer could choose to impose different
video limits on the left center and right cameras, but it would be within
its rights to do so, just as the provider would then be within its rights to
send them all at the lowest of the 3 levels.

Regards,

Andy

(As co-chair)

*If* this works and satisfies all objectives then it will be a helpful
simplification. But it may be that simplifying things this way excludes some
required behavior.

        Thanks,
        Paul

Hi,

all belong to one encoding group, here my view is "no" since they have

Thanks

Roni Even

---

From: "Roni Even" <[email-masked]>
Date: Mon, 5 Nov 2012 09:00:34 -0500

Hi Andy,

I was wondering how can an MCU create a common encoding group and individual
encodes if all participating EP do not have a common limit since it needs to
provide a common advertisement

Roni

 

individual encoding.

 

 

 

Regards,

Andy

(As co-chair)

        Thanks,
        Paul

Hi,

Thanks

Roni Even

---

From: Andy Pepperell <[email-masked]>
Date: Tue, 6 Nov 2012 12:01:11 +0000

Hi Roni,

Media Captures.

I'll have a go...

Firstly, I'd say that the dynamic mapping relates to *capture encodings*
rather than media captures. You mention the MCU case; I'd say this depends
on how you envisage the MCU case operating:

If the MCU were transcoding then it would offer separate capture scene
entries for 1 capture, 2 capture, 3 capture etc. transcodings of the main
scene, and perhaps a capture scene for presentation as well.

If the MCU were switching, I'd envisage it offering some number of loudest
speaker streams in a "main" capture scene and allowing consumer devices to
choose the number they were able to render and display.

I myself never saw the multipoint case as involving all individual
participants' advertisements being sent out to all other participants - I
think MCUs should be performing some sensible aggregation of what
participants advertise to them in order to avoid scalability problems.

I agree that if you're switching from multiple sources and not transcoding
then all of those contributing provider advertisements might contribute to
(/ modify) the MCU's provider advertisement that gets sent out, though I
don't really see how dynamic RTP vs static RTP is that relevant here (i.e.
I think the complexities here are present no matter how the SSRCs etc. are
derived). In the dynamic case, the consumer essentially chooses header
extension IDs for the capture encodings it requests from the provider, and
decodes / renders the media sent to it from the provider.

I wonder if we're getting 2 separate topics needlessly entangled here, a)
dynamic vs static mapping and b) how a multipoint provider determines its
own provider advertisement from multiple participants' such advertisements.
For the multipoint case, I'd argue that the MCU's role here would be to
distil down the individual other participants' media into sensible capture
scenes and capture scene entries that give each consumer a sensible set of
media captures to choose from.

Regards,

Andy

On Mon, Nov 5, 2012 at 1:55 PM, Roni Even <[email-masked]> wrote:

> Andy,****
> Media Captures. ****
> limits may enforce different limitations.****
>
> ** **
>
> In the static mapping case all this can be achieved using imageattr and
> the codec parameters****
>
> Roni****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Andy Pepperell
> *Sent:* 02 November, 2012 11:20 AM
> *To:* Paul Kyzivat
> *Cc:* [email-masked]
> *Subject:* Re: [clue] Do we need the encodings as secified in the
> framework document - proposal to remove them from the framework****
>
> ** **
>
> >Paul:****
>
> some required behavior.****
>
> ** **
>
> I certainly agree with Paul's point here... however, with not all that
> many of Roni's (as might be expected :) ) - to be specific:****
>
> ** **
>
> >Roni:****
>
> Yes, I think that's a good summary.****
>
>  ****
>
> Also agreed - apart from the H.264 point I think - a maximum resolution
> and a maximum frame rate doesn't necessarily imply a maxMbps - many systems
> can encode 1280x720 at some frame rate < 30 *or* some lower resolution at
> up to 30fps without necessarily being able to send 720p30, surely?****
>
>  ****
>
>  ****
>
> > Roni:****
>
> Telepresence architectures we see today:****
>
> their "1 screen" products linked together)****
>
> 2) a flexible media generation system such as a software or hardware MCU**
> **
>
> ** **
>
> for its left constituent, one for its centre and one for its right.****
>
> ** **
>
> >Roni:****
>
> ** **
>
> media capture, each individual encoding can be used by one media capture,
> the decision of which one to use according to the framework is by the
> provider. The consumer does not select an individual encode.****
>
> ** **
>
> this well (though Mark deserves the credit for that, obviously!).****
>
> ** **
>
> >Roni: ****
>
> in this way).****
> >Roni:****
> consumer has greater flexibility in which it can choose to be sent to it.*
> ***
>
> ** **
>
> ** **
>
> >Roni: ****
>
> >The video is H.264 specific and not general****
>
> ** **
>
> The problem we had was that we needed to come up with a way of describing
> a big block of video encoding capability and how it could be subdivided.
> e.g. you might be able to send 2 x 720p30 but not a single 720p60. The best
> way.****
> >Roni:****
> not be fully understanding the intricacies of the case you describe).****
>
> ** **
>
> only three streams, the individual encodes limit will be low so he will not
> be able to use the full compute resources.****
>
> ** **
>
> >Roni:****
>
> itself a useful representation of the scene).****
>
>  ****
>
> different priority and should not compete for compute resources.****
>
> ** **
>
> the consumer to allocate encodings to the media captures).****
>
> ** **
>
> >Roni:****
>
> within its rights to send them all at the lowest of the 3 levels.****
>
> ** **
>
> >Roni: ****
>
> >Roni: ****
> text that defines how to use it for configuring encoding to media captures
> including for the simulcast case****
>
> ** **
>
> more to discuss on this...****
>
> ** **
>
> Regards,****
>
> ** **
>
> Andy****
>
> ** **
>
> ** **
>
> On Thu, Nov 1, 2012 at 4:38 PM, Paul Kyzivat <[email-masked]>
> wrote:****
>
> (As co-chair)
>
>         Paul****
> On 10/31/12 5:49 AM, Roni Even wrote:****
>
> Hi,
>
> ****
> https://www.ietf.org/mailman/listinfo/clue****
>
>
> https://www.ietf.org/mailman/listinfo/clue****
>
> ** **
>

---

From: Andy Pepperell <[email-masked]>
Date: Tue, 6 Nov 2012 12:32:52 +0000

Hi Roni,

>I was wondering how can an MCU create a common encoding group and
individual encodes if all participating EP do not have a common limit since
it needs to provide a common advertisement

Again, I'd say there's no single algorithm here as it would depend on the
characteristics of the MCU - whether it is transcoding or switching, and
what experience it's intending to provide to other (presumably
CLUE-capable) participants acting as consumers.

It is true though that the MCU needs to create a provider advertisement
that in some senses summarizes the conference as a whole; however, it's
also the case that:
- each conference participant will have performed their own individual O/A
exchange with the MCU to set session-wide limits for things such as overall
H.264 maxMbps and to agree on which payload types to use for different
encoding types
- the provider advertisement values are strictly maximum values, and so the
MCU does not need to automatically decrease its advertised values to all
consumers based on lower values in other participants' advertisements.

If we're talking about a non-transcoding MCU which was intending to provide
just a single loudest speaker video stream to each consumer, I'd expect
that MCU's provider advertisement to include a capture scene entry of a
single video capture whose encoding group and encoding would reflect the
largest bandwidth / resolution / maxMbps of all potential sources for that
capture (but need not; this could perfectly reasonably be capped by a limit
chosen by the MCU). Once a consumer had requested a capture encoding of
that media capture, it might be (depending on the values of the encoding
limits specified by the consumer) that the MCU might issue new
configuration messages to other participants with lower maximum values in
order that that capture can be switched (with appropriate payload
re-writing / header extension additions) to multiple consumers. This would
be subject to individual, per-consumer, tweaking; as well as the payload
re-writing, it might be that the MCU would send the "2nd loudest" speaker
back to the loudest speaker, or might eschew additional offer / answer
cycles to all participants to accommodate a new consumer unable to receive
the same set of video codecs as the others at the expense of a slightly
worse conferencing experience for those lower capable consumers, for
instance.

Going back to:
>I was wondering how can an MCU create a common encoding group and
individual encodes if all participating EP do not have a common limit since
it needs to provide a common advertisement

I think I'd be reluctant to go into too much more detail on this here
because I think perhaps the question needs restating slightly - a common
encoding group and individual encodes would be reasonably straightforward
to come up with given the semantics of those encodings' values as being
maximum values for bandwidth / resolution / mbps etc. Perhaps a more
interesting question is how an MCU would modify the configuration messages
it sends to other conference participants based on new participants'
configuration messages to it. In many senses though I would say these
issues aren't introduced by CLUE - a straightforward SIP MCU today that was
trying to switch loudest speaker streams to conference participants when
those participants support different codecs / bandwidth limits would run
into very similar "lowest common denominator" type issues, and presumably
may need to readvertise to its participants, rewrite RTP payload types etc.
as part of being able to switch one source stream out to multiple
destinations...

Regards,

Andy

P.S. You say "...  since it needs to provide a common advertisement" -
technically, the MCU would be perfectly within its rights to form a new
provider advertisement for each consumer / participant. This might be
appropriate, for instance, if it wished to reflect the overall call
bandwidth in its individual encodings or to, say, omit H.264 parameters if
the initial offer answer yielded no common H.264 modes, or to omit video
captures if its video capacity had been exhausted.

On Mon, Nov 5, 2012 at 2:00 PM, Roni Even <[email-masked]> wrote:

> Hi Andy,****
>
> I was wondering how can an MCU create a common encoding group and
> individual encodes if all participating EP do not have a common limit since
> it needs to provide a common advertisement****
>
> Roni****
>
> ** **
>