Subject: Re: [MMUSIC] [rtcweb] Quick comments on draft-roach-rtcweb-glareless-add-00

From: Stefan Håkansson LK <[email-masked]>
Date: Fri, 10 May 2013 06:37:21 +0000

On 5/8/13 11:19 PM, Adam Roach wrote:
> On 5/8/13 15:07, Paul Kyzivat wrote:

>> I remain dubious that this glare situation is a problem that needs to
>> be fixed.
>
> So am I, but Justin seems to be obsessed with it. That's why I'm
> proposing a non-normative recipe that allows people who are
> unnecessarily preoccupied with the issue to indulge their peccadillos.
> The rest of us are free to ignore it.

I think that glare coming from simultaneous add or remove of media may 
not be that common.

But I can easily think of other situations that would lead to glare for 
Plan A.

Just as an example, think about a conference scenario with a centralized 
node. Now, one of the participants wants to mute her/his audio, and as a 
result the audio m-line would go from sendrecv to recvonly. At roughly 
the same time the server wants to disable the high res video and enable 
the low res video (because this user will be moved, based e.g. on the 
recent audio activity, from being displayed in the main video window to 
be in a thumbnail on the screen of the other participants in the 
conference), so it wants to manipulate the direction attribute of other 
m-lines.

So there would be two offers sent at roughly the same time, which means 
we would have glare.

I think the result would be similar if the server would instead ask for 
another resolution using the imageattr.

You could argue that all control should be left to one end-point, but it 
is sort of clumsy.

Plan B, with the individual control of sources, seems better fitted to 
handle this kind of situations.

> /a

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 10 May 2013 17:20:02 -0400

On 5/10/13 2:37 AM, Stefan Håkansson LK wrote:
...
> So there would be two offers sent at roughly the same time, which means

What do you mean by "roughly"?

How long does it take to do an O/A exchange that doesn't require human 
input? Presumably less than a second, maybe considerably less. (Or will 
it be more with RTCWEB and web servers in the path?)

What is the probability that the other side, for independent reasons, 
decides to make a change within that interval? (And it isn't really the 
whole interval - it is only the first part of it.)

There may indeed be realistic cases for glare, but they will probably be 
because both changes are keyed to a common event.

	Thanks,
	Paul

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sat, 11 May 2013 11:50:03 +0000

> it be more with RTCWEB and web servers in the path?)

That sounds right to me.

> whole interval - it is only the first part of it.)

Given that in multiparty services the "main" video changes quite often, 
I'd not say this is extremely low. It will happen.

But, given that there is a way to roll-back in a predictable way I think 
glare resolution could be quick - nothing stops the web application from 
including a random number in every offer, and resolve glare based on 
that (one of the offers "win").

So I think glare will happen in practice (that is what I wanted to point 
out), but it is not a show-stopper provided there is a way to roll back.

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 11 May 2013 12:24:09 -0400

> I'd not say this is extremely low. It will happen.

Are you thinking that there will be an SDP change every time the speaker 
changes???

Or do you mean changes when participants join and leave a conference?

I would hope there will normally not be SDP changes for either of those 
events.

> out), but it is not a show-stopper provided there is a way to roll back.

I'm sure it will happen. The question is whether it happens often enough 
to do more than count on the normal glare recovery procedures.

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 11 May 2013 20:17:35 -0700

>> So there would be two offers sent at roughly the same time, which means

There are implementations where no signalling is required for the mixer to go from high res to lower res. For example, if the receiver declares what it can receive and the mixer selects the simulcast stream for that receiver based on capabilities and conditions, with the stream properties declared in band.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Sun, 12 May 2013 17:18:23 +0000

> Or do you mean changes when participants join and leave a conference?

I meant the former (at speaker change).

I guess that different designs can be chosen for media from central node 
to end-user clients, and some of them (as Bernard pointed out) would not 
require any signaling. Still, there might be cases where signaling is 
needed.

But leaving that aside, we still have the case of the media end-user 
client -> central node. If you're serious about not wasting bandwidth 
(and with my radio/mobile background I am sort of obsessed :-) ) all 
end-user clients which do not end up as main video anywhere should only 
transmit a low resolution video. So at every speaker change there would 
be at least one client that is asked to stop sending high resolution and 
only transmit low resolution, and one client that is asked to start 
transmitting high resolution video.

I think that this would best be done using some RTCP signaling scheme, 
but I also think there was an agreement a long time ago that SDP O/A 
should be used. In this case either to transmit a new imageattr value 
requesting another resolution, or to pause/resume - using recv on/off in 
Plan B or the direction attribute in Plan A - the low and high 
resolution simulcast streams accordingly (or I assume to pause/resume 
the enhancement layers if a scalable codec is used).

> I would hope there will normally not be SDP changes for either of those

I would like to avoid them too, but I'm not sure how.

---

From: Emil Ivov <[email-masked]>
Date: Sun, 12 May 2013 21:28:50 +0300

> require any signaling. Still, there might be cases where signaling is

Right, however in cases where signalling is indeed needed, should it
really be defined by RTCWEB rather than the application developer?

> I think that this would best be done using some RTCP signaling scheme,

RTCP is just one of the possibilities. This could also be implemented
in a number of other ways that would all be more efficient for the
specific application and all out-of-scope for rtcweb.

> should be used.

Well the way I remember it SDP O/A was chosen for session negotiation
and that other forms of signalling were explicitly left out of the
charter.

> In this case either to transmit a new imageattr value

imageattr is great as a declarative parameter indicating capabilities:
"I can't show more than 720p so there's no point to send me more than
that". Using it as a way to switch between thumbnail and HQ videos is
not entire impossible but is not particularly efficient either.

> I would like to avoid them too, but I'm not sure how.

Well we can either figure that out (and it seems that we can't quite
agree on a solution) or make sure APIs have all it takes for the
matter to be resolved by app-to-app signalling.

Emil

--
https://jitsi.org

---

From: Stefan Håkansson LK <[email-masked]>
Date: Mon, 13 May 2013 09:21:46 +0200

> and that other forms of signalling were explicitly left out of the

I took the result of the discussion and poll at the IETF-84 
(http://tools.ietf.org/wg/rtcweb/minutes?item=minutes-84-rtcweb.html) as 
an indication of that SDP O/A should be used for this kind of in-session 
changes as well.

> not entire impossible but is not particularly efficient either.

To me it looks like a solution that people might feel inclined towards 
using.

But there are others that want to use simulcast, and then pause/resume 
sending of high and low resolution streams, and that problem is more or 
less the same when it comes to signaling.

And even if we omitted the possibility for a receiver to indicate to the 
sender (via standardized signaling) that it should pause or resume 
sending, and instead left that up to an API (at sender side) and 
proprietary signaling, we still have to have some kind of signaling IMO: 
the sender can not just stop sending RTP packets, there need to be a 
signal to the receiver so it understands this (and does not think there 
are extreme packet losses or some other errors) and can indicate to the 
application at the receiving end (in WebRTC it should result in a "mute" 
event being fired on the PC-track in question).

I think the current assumption is SDP O/A for that kind of signaling.

---

From: Emil Ivov <[email-masked]>
Date: Mon, 13 May 2013 17:33:10 +0300

Hey Stefan,

> an indication of that SDP O/A should be used for this kind of in-session 

True, I am not sure we all realized the number of use cases this would
end up encompassing and how we would end up implementing signalling
features into SDP.

> signal to the receiver so it understands this 

I never implied there should be no signalling. Just that implementing it
all over SDP O/A leads to unforeseen complexities.

RTCP BYE and/or SSRC expiration would be one option here. Custom
signalling another.

Cheers,
Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 13 May 2013 10:44:18 -0400

I'm not going to comment further on the specifics here, since they have 
been well flogged already. But the point of this particular discussion 
was about the likelihood of glare. Stefan's assertion was that O/As 
initiated by one end due to rapid changes in which streams it wants to 
receive will drive up the possibility of glare when the other end wants 
to do an O/A.

I'll just say that if the number of O/As has gone up that much, then 
that was a bad choice of mechanism.

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 13 May 2013 08:37:05 -0700

I think you are right on target - but if we do not have a clear articulation of the line between RTCP and O/A, then it is going to be hard to prevent going off Into the weeds.

On May 13, 2013, at 7:44, "Paul Kyzivat" <[email-masked]> wrote:

> I'm not going to comment further on the specifics here, since they have been well flogged already. But the point of this particular discussion was about the likelihood of glare. Stefan's assertion was that O/As initiated by one end due to rapid changes in which streams it wants to receive will drive up the possibility of glare when the other end wants to do an O/A.
> 
> I'll just say that if the number of O/As has gone up that much, then that was a bad choice of mechanism.
> 
>    Thanks,
>    Paul
> 
>

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Mon, 13 May 2013 17:28:35 +0000

We already have RTCP to pause / un pause things and it seems like we should have RTCP to change resolution, frame rate, bandwidth inside of the parameters set in the SDP envelope (some people think we have that ). I agree that needs to all be clearly specifies how one does all of that with RTCP and how an endpoint signals in in SDP that it is capable of such negotiations. It seems fairly easy to specify whatever parts of that are missing. 

On May 13, 2013, at 9:37 AM, Bernard Aboba <[email-masked]> wrote:

> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 13 May 2013 10:58:49 -0700

> I would like to avoid them too, but I'm not sure how.

[BA] It is probably not possible to avoid them in all cases, but we can define RTCP functionality which along with some set of functionality (e.g. Simulcast, layered coding) will minimize them. This is worth doing (and soon).

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 13 May 2013 11:01:06 -0700

I agree with this, and think the overall experience will be better if we can do this.  It should have been defined years ago.

---

From: Stefan Håkansson LK <[email-masked]>
Date: Tue, 14 May 2013 10:24:24 +0200

On 2013-05-13 19:58, Bernard Aboba wrote:
>
>
> On May 12, 2013, at 10:18, "Stefan Håkansson LK"
>>> I would hope there will normally not be SDP changes for either of
>>> those events.
>>
> [BA] It is probably not possible to avoid them in all cases, but we
> can define RTCP functionality which along with some set of
> functionality (e.g. Simulcast, layered coding) will minimize them.
> This is worth doing (and soon).

I agree to this!

---

From: Emil Ivov <[email-masked]>
Date: Tue, 14 May 2013 12:17:14 +0300

>> This is worth doing (and soon).

RTCP `certainly sounds a lot better than SDP O/A

Emil

-- 
https://jitsi.org