Subject: [MMUSIC] pause/resume in draft-ietf-mmusic-sdp-simulcast

From: Matthew Kaufman <[email-masked]>
Date: Tue, 9 Jul 2019 23:09:15 +0000

Issue 1 - Shouldn't stream pause/resume be required?

If I'm a SFM and you are sending me multiple simulcast streams and I decide that nobody is watching one of them, it is wasteful of bandwidth and power (CPU) for you to continue sending a stream I don't want. Likewise if I'm a client and you're sending me simulcast streams directly. I believe that for this reason, pause/resume should be mandatory in this draft. All of the below issues rely on having pause/resume messaging for their successful resolution.

Furthermore, I believe that we must require that TMMBR 0 NOT be used for pause of simulcast streams, because of the relevant note in section 5.5 of RFC 7728. (See Issue 3 below)

Issue 2 - What to send in the media (or signaling) layer when bitrate adaptation says there's not enough bandwidth to send?

If I'm a SFM and I'm receiving your three streams that you are sending (ex., 180P, 480P, 720P all with temporal scaling) and you decide that you can't send the 720P any more, how can I tell the difference between "sender ran out of bandwidth" and "sender is broken"? In particular, if I have subscribers to your 720P stream, and you ran out of bandwidth, I need to know ASAP to start forwarding them the 480P stream instead. Thus an indication that you have paused, preferably in the media stream itself, would be useful... and furthermore, it would be nice to know why you've chosen to do that.

This can be (largely) solved by requiring senders to use the PAUSED indication for this case, and calling that out explicitly in section 7.1 (or elsewhere) of the draft (though as a receiver, it would be nice to know the difference between the various reasons a sender might have appeared to me to have unilaterally decided to pause)

Issue 3 - What to send when bitrate adaptation says there's not enough bandwidth to send *and* pause has been requested for a lower bandwidth stream?

If I'm a SFM and I'm receiving your three streams (ex., 180P, 480P, and 720P) and I note that my viewers are only watching your 720P it would save bandwidth and CPU to send you a pause for the 180P and 480P streams. Once I request that of you, and these are paused, what are you allowed to do when bandwidth adaptation determines that there isn't sufficient bandwidth to send the 720P? Can you unilaterally un-pause the 480P stream in order to keep sending video, or are you forced to stop the 720P and hope the SFM guesses properly that it needs to request un-pause of the 480P?

Likewise, what if the lower resolution streams were signaled as initially paused? Can the sender unilaterally un-pause one in order to use it instead of a higher-bandwidth choice?

I believe the answer here lies in RFC 7728, where an RTP sender may due to local considerations choose to resume at any time, however because TMMBR 0 prohibits this, TMMBR 0 must not be used to pause simulcast streams that a remote sender may need to begin using in order to bandwidth-adapt.

This required behavior (unilateral unpause if necessary) should be noted in section 7.1 of the draft.

Matthew Kaufman

---

From: Bernard Aboba <[email-masked]>
Date: Wed, 10 Jul 2019 21:16:47 -0700

RFC 7728 (Pause/Resume) states the problem as follows:

   "This document describes several
   scenarios in multimedia communication where a conferencing node or
   participant chooses to temporarily pause an incoming RTP [RFC3550
<https://tools.ietf.org/html/rfc3550>]
   stream and later resume it when needed....

   This becomes especially useful when the media sources are
   provided in multiple encoding versions (Simulcast) [SDP-SIMULCAST
<https://tools.ietf.org/html/rfc7728#ref-SDP-SIMULCAST>] or
   with Multi-Session Transmission (MST) of scalable encoding such as
   Scalable Video Coding (SVC) [RFC6190 <https://tools.ietf.org/html/rfc6190>]."

There are several issues with this problem statement (and the solution):

1. The problem statement does not cover situations in which simulcast
or SVC is encoded with a single SSRC

(both are possible with AV1, for example).

2. The solution does not provide information equivalent to H.264's SEI
layer drop message. For example,

a "PAUSED" indication does not indicate the last sequence number sent
prior to pausing the stream.

Therefore the SFU still may need to NACK and wait for retransmission
of packets that were

never sent.

So as to broaden the problem space and provide a better solution, a
different approach is needed.

Hopefully that solution (under discussion in AoMedia) will have a
better chance of being deployed than RFC 7728.

On Tue, Jul 9, 2019 at 4:09 PM Matthew Kaufman <[email-masked]>
wrote:

> If I'm a SFM and you are sending me multiple simulcast streams and I
> decide that nobody is watching one of them, it is wasteful of bandwidth and
> power (CPU) for you to continue sending a stream I don't want. Likewise if
> I'm a client and you're sending me simulcast streams directly. I believe
> that for this reason, pause/resume should be mandatory in this draft. All
> of the below issues rely on having pause/resume messaging for their
> successful resolution.
>
> Furthermore, I believe that we must require that TMMBR 0 NOT be used for
> pause of simulcast streams, because of the relevant note in section 5.5 of
> RFC 7728. (See Issue 3 below)
>
> Issue 2 - What to send in the media (or signaling) layer when bitrate
> adaptation says there's not enough bandwidth to send?
>
> If I'm a SFM and I'm receiving your three streams that you are sending
> (ex., 180P, 480P, 720P all with temporal scaling) and you decide that you
> can't send the 720P any more, how can I tell the difference between "sender
> ran out of bandwidth" and "sender is broken"? In particular, if I have
> subscribers to your 720P stream, and you ran out of bandwidth, I need to
> know ASAP to start forwarding them the 480P stream instead. Thus an
> indication that you have paused, preferably in the media stream itself,
> would be useful... and furthermore, it would be nice to know why you've
> chosen to do that.
>
> This can be (largely) solved by requiring senders to use the PAUSED
> indication for this case, and calling that out explicitly in section 7.1
> (or elsewhere) of the draft (though as a receiver, it would be nice to know
> the difference between the various reasons a sender might have appeared to
> me to have unilaterally decided to pause)
>
> Issue 3 - What to send when bitrate adaptation says there's not enough
> bandwidth to send *and* pause has been requested for a lower bandwidth
> stream?
>
> If I'm a SFM and I'm receiving your three streams (ex., 180P, 480P, and
> 720P) and I note that my viewers are only watching your 720P it would save
> bandwidth and CPU to send you a pause for the 180P and 480P streams. Once I
> request that of you, and these are paused, what are you allowed to do when
> bandwidth adaptation determines that there isn't sufficient bandwidth to
> send the 720P? Can you unilaterally un-pause the 480P stream in order to
> keep sending video, or are you forced to stop the 720P and hope the SFM
> guesses properly that it needs to request un-pause of the 480P?
>
> Likewise, what if the lower resolution streams were signaled as initially
> paused? Can the sender unilaterally un-pause one in order to use it instead
> of a higher-bandwidth choice?
>
> I believe the answer here lies in RFC 7728, where an RTP sender may due to
> local considerations choose to resume at any time, however because TMMBR 0
> prohibits this, TMMBR 0 must not be used to pause simulcast streams that a
> remote sender may need to begin using in order to bandwidth-adapt.
>
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 11 Jul 2019 08:13:58 +0000

Hi Bernard,

On Wed, 2019-07-10 at 21:16 -0700, Bernard Aboba wrote:
>    participant chooses to temporarily pause an incoming RTP [RFC3550]
>    provided in multiple encoding versions (Simulcast) [SDP-SIMULCAST] 
> or
>    Scalable Video Coding (SVC) [RFC6190]."
> 

Packetizing stand alone simulcast streams in the same SSRC appears
counter productive if one targets RTP and failing to utilize the
mechanism that exists. 

For Scalable encoding the sub-stream control still are lacking in
support if not the current codec control messages can be used combined
with bandwidth limiations to indicate the desired sub-set of streams.
And for an SFU that might want two different branches of the tree of
scalability information there clearly are need for a solution if one
insist on not using the mechanisms of RTP that exist and actually split
it out over multiple SSRCs. 

> 
> 2. The solution does not provide information equivalent to H.264's
> SEI layer drop message. For example,

I think you need to re-read RFC 7728. Section 8.2 (PAUSED) says:

   PAUSED SHALL contain a fixed-length 32-bit parameter at the start of
   the Type Specific field with the extended RTP sequence number of the
   last RTP packet sent before the RTP stream was paused, in the same
   format as the extended highest sequence number received
   (Section 6.4.1 of [RFC3550]).

I assume you where thinking of TMMBR/TMMBN which doesn't contain this.
A reason for choosing PAUSE/PAUSED. 

Cheers

Magnus Westerlund

> On Tue, Jul 9, 2019 at 4:09 PM Matthew Kaufman <
> [email-masked]> wrote:
> > If I'm a SFM and you are sending me multiple simulcast streams and
> > I decide that nobody is watching one of them, it is wasteful of
> > bandwidth and power (CPU) for you to continue sending a stream I
> > don't want. Likewise if I'm a client and you're sending me
> > simulcast streams directly. I believe that for this reason,
> > pause/resume should be mandatory in this draft. All of the below
> > issues rely on having pause/resume messaging for their successful
> > resolution.
> > 
> > Furthermore, I believe that we must require that TMMBR 0 NOT be
> > used for pause of simulcast streams, because of the relevant note
> > in section 5.5 of RFC 7728. (See Issue 3 below)
> > 
> > Issue 2 - What to send in the media (or signaling) layer when
> > bitrate adaptation says there's not enough bandwidth to send?
> > 
> > If I'm a SFM and I'm receiving your three streams that you are
> > sending (ex., 180P, 480P, 720P all with temporal scaling) and you
> > decide that you can't send the 720P any more, how can I tell the
> > difference between "sender ran out of bandwidth" and "sender is
> > broken"? In particular, if I have subscribers to your 720P stream,
> > and you ran out of bandwidth, I need to know ASAP to start
> > forwarding them the 480P stream instead. Thus an indication that
> > you have paused, preferably in the media stream itself, would be
> > useful... and furthermore, it would be nice to know why you've
> > chosen to do that.
> > 
> > indication for this case, and calling that out explicitly in
> > section 7.1 (or elsewhere) of the draft (though as a receiver, it
> > would be nice to know the difference between the various reasons a
> > sender might have appeared to me to have unilaterally decided to
> > pause)
> > 
> > Issue 3 - What to send when bitrate adaptation says there's not
> > enough bandwidth to send *and* pause has been requested for a lower
> > bandwidth stream?
> > 
> > If I'm a SFM and I'm receiving your three streams (ex., 180P, 480P,
> > and 720P) and I note that my viewers are only watching your 720P it
> > would save bandwidth and CPU to send you a pause for the 180P and
> > 480P streams. Once I request that of you, and these are paused,
> > what are you allowed to do when bandwidth adaptation determines
> > that there isn't sufficient bandwidth to send the 720P? Can you
> > unilaterally un-pause the 480P stream in order to keep sending
> > video, or are you forced to stop the 720P and hope the SFM guesses
> > properly that it needs to request un-pause of the 480P?
> > 
> > Likewise, what if the lower resolution streams were signaled as
> > initially paused? Can the sender unilaterally un-pause one in order
> > to use it instead of a higher-bandwidth choice?
> > 
> > I believe the answer here lies in RFC 7728, where an RTP sender may
> > due to local considerations choose to resume at any time, however
> > because TMMBR 0 prohibits this, TMMBR 0 must not be used to pause
> > simulcast streams that a remote sender may need to begin using in
> > order to bandwidth-adapt. 
> > 
-- 
Cheers

Magnus Westerlund 

----------------------------------------------------------------------
Network Architecture & Protocols, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------