Subject: [MMUSIC] Multicast and one-m-line-per-stream (Re: Thoughts on m= lines and media streams)

From: Harald Alvestrand <[email-masked]>
Date: Sun, 03 Feb 2013 00:13:06 +0100

Adam,

I hate to play the multicast card, because I'm fundamentally 
uninterested in multicast at the moment, but....

in your scheme 2, how would you represent multicast and transport-mixer 
based sessions?

On 02/01/2013 01:17 AM, Adam Roach wrote:
> Based on the conversations we've had so far, I've put together the 
> following summary of my thoughts on the topic of whether we want to 
> put multiple media streams in a single m= line.
>
> http://www.ietf.org/id/draft-roach-mmusic-mlines-00.txt
>
> Sorry for the relatively short lead time before next week's meeting -- 
> I had meant to put this kind of summary together a week or two ago, 
> but have been far busier than I anticipated.
>
> /a
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Justin Uberti <[email-masked]>
Date: Sat, 2 Feb 2013 22:04:47 -0800

The prior commenters captured some of my concerns already, but here's a
quick summary of issues I noticed while reading over this draft:

1) "1a" and "2", as described, have fatal problems if the receiver rejects
the m= line that contains all the attributes (e.g. a=ice-ufrag, a=crypto,
etc)

2) In "1b", it's not clear what the purpose of the "a=group:MMT foo bar
zoe" line is.

3) Proposal "2" does not include the needed a=ssrc lines for demultiplexing
the received streams. IOW, proposal 2 requires RFC 5576 just like 1a and 1b.

4) The codec selection argument in Section 2.3.1 feels like a "Red Herring"
to me. Practically speaking, I think there are few, if any, real-world
settings where one might want to use a codec prioritization mechanism to
ask for a low-bitrate codec for stream A and a high-bitrate codec for
stream B (where the codecs used are different). A more realistic example
would be where one might want a lower bitrate  for a specific multi-rate
codec (e.g. Opus) to be used for stream A, and higher bitrate for stream B.
This of course can be handled with the per-ssrc fmtp attributes that are
specified in RFC 5576. However, I think in almost all cases, the receiver
is in a much worse position to make these decisions that the sender; the
sender, aware of the estimated bandwidth to the receiver, and the relative
value/complexity of its streams, can make more informed decisions about
bandwidth allocation, and perform them without any signaling delay.

5) The directional attribute discussion in Section 4 omits one important
consequence of the fact that a=ssrc indications are unidirectional, namely
that streams could be added or removed by both endpoints simultaneously
without causing a glare condition. m= lines, on the other hand, since they
are identified by their index, do not have this property; if both sides
decide to add a m= line at the same time, bad things happen.

6) The legacy interop argument in Section 4 feels a bit hand-wavy. Aside
from the one case of having video + presentation streams, I have not yet
heard of widely deployed equipment that uses multiple audio or video m=
lines, and certainly any such equipment will not understand BUNDLE or MMT.
Therefore a gateway is required, and any gateway that was performing BUNDLE
or MMT could certainly do the minor additional work to package the SDP as 1
m= line (remember, it already needs to add the a=ssrc attributes). Ergo, I
don't agree that the general case of N m= lines needs to be handled by all
RTCWEB implementations.

In summary, I see option "2" as a "1b" variant that comes with a more
verbose and redundant syntax, several shortcomings (#1 and #5 above), and
questionable upside.

-j-

On Thu, Jan 31, 2013 at 11:48 PM, Stach, Thomas <
[email-masked]> wrote:

>
> Adam,
>
> thanks for analysing the various proposals and putting the draft together.
> From a quick read I have some issues with backward compatibility of your
> proposal to include
> "common" attributes only in the first m-section.
> Assuming that the examples in the draft are offers, an answerer that
> didn't implement any of the mechanisms would probably show the
> following behavior.
>
> Alternativ 1a and 2 have ICE candidates only for the first m-section.
> An answerer that doesn't understand the grouping semantics could -
> dependent on the network topology - end up having audio, but no video
> connectivity.
> With having other attributes only in the first m-section we could get
> other issues.
>
>
> Alternative 1b doesn't have this problem, but the offer would most likely
> be rejected resulting in no media at all.
> I preferred the original mmt proposal as it had additional m-sections,
> that could be accepted by a legacy answerer without the need for a GW that
> performs complex SDP operations.
>
>
> Regards
> Thomas
>
>
> Adam Roach wrote:
> > following summary of my thoughts on the topic of whether we
> > want to put
> > multiple media streams in a single m= line.
> >
> > Sorry for the relatively short lead time before next week's
> > meeting -- I
> > had meant to put this kind of summary together a week or two ago, but
> > have been far busier than I anticipated.
> >
> > /a

---

From: Adam Roach <[email-masked]>
Date: Tue, 05 Feb 2013 09:32:52 -0500

On 2/3/13 01:04, Justin Uberti wrote:
> 1) "1a" and "2", as described, have fatal problems if the receiver 
> rejects the m= line that contains all the attributes (e.g. 
> a=ice-ufrag, a=crypto, etc)

Yeah, this part of the solution was put together kind of off-the-cuff. 
I'd considered three approaches, and this one was the least verbose. My 
second choice, which has better backwards compatibility properties 
anyway, is mandating that the transport-level attributes are in all the 
m= sections to which they apply. You end up with redundant information, 
but it does get around issues with rejecting some sections.

> 2) In "1b", it's not clear what the purpose of the "a=group:MMT foo 
> bar zoe" line is.

Unfortunately, I really didn't have the time I wanted to put this draft 
together, and there's more than a little carelessness in the SDP 
examples. That shouldn't be there.

> 3) Proposal "2" does not include the needed a=ssrc lines for 
> demultiplexing the received streams. IOW, proposal 2 requires RFC 5576 
> just like 1a and 1b.

And this is the sloppiness I referred to above. The different sections 
should have non-overlapping RTP payload types.

> 4) The codec selection argument in Section 2.3.1 feels like a "Red 
> Herring" to me. Practically speaking, I think there are few, if any, 
> real-world settings where one might want to use a codec prioritization 
> mechanism to ask for a low-bitrate codec for stream A and a 
> high-bitrate codec for stream B (where the codecs used are different). 
> A more realistic example would be where one might want a lower bitrate 
>  for a specific multi-rate codec (e.g. Opus) to be used for stream A, 
> and higher bitrate for stream B. This of course can be handled with 
> the per-ssrc fmtp attributes that are specified in RFC 5576. However, 
> I think in almost all cases, the receiver is in a much worse position 
> to make these decisions that the sender; the sender, aware of the 
> estimated bandwidth to the receiver, and the relative value/complexity 
> of its streams, can make more informed decisions about bandwidth 
> allocation, and perform them without any signaling delay.

You're asserting that the person sending the media (to whom the quality 
has little or no impact) is a better party to select the quality of a 
stream than the person receiving it?

I have nothing to say to that. It's sufficiently false on its face that 
I really have nowhere to start grappling with it.

> 5) The directional attribute discussion in Section 4 omits one 
> important consequence of the fact that a=ssrc indications are 
> unidirectional, namely that streams could be added or removed by both 
> endpoints simultaneously without causing a glare condition. m= lines, 
> on the other hand, since they are identified by their index, do not 
> have this property; if both sides decide to add a m= line at the same 
> time, bad things happen.

None of the three approaches avoids the necessity to handle glare. With 
a stateful offer/answer approach, there's simply no way around it. We 
would have to go back to the drawing board if we wanted to come up with 
a system that avoided glare (and even then, I don't think success is 
very likely).

> 6) The legacy interop argument in Section 4 feels a bit hand-wavy. 
> Aside from the one case of having video + presentation streams, I have 
> not yet heard of widely deployed equipment that uses multiple audio or 
> video m= lines, and certainly any such equipment will not understand 
> BUNDLE or MMT. Therefore a gateway is required, and any gateway that 
> was performing BUNDLE or MMT could certainly do the minor additional 
> work to package the SDP as 1 m= line (remember, it already needs to 
> add the a=ssrc attributes). Ergo, I don't agree that the general case 
> of N m= lines needs to be handled by all RTCWEB implementations.

Truth be told, even though I called this "legacy interop," it's more 
accurate to think of it as "interop with non-RTCWEB implementations that 
use SDP," of which legacy is an important subset.

In any case, if we're jumping to needing a gateway to do substantial SDP 
rewrites, then I question the use of SDP at all in RTCWEB. The selection 
of SDP, to my understanding, was to provide simpler interop with 
non-RTCWEB implementations. Once we assume that you're necessarily 
rewriting it for such interop to work, we may as well abandon it altogether.

/a

---

From: Adam Roach <[email-masked]>
Date: Tue, 05 Feb 2013 10:52:09 -0500

Before I respond, I do want to make sure that this is the same Harald 
Alvestrand who was proposing that we dismiss the vast majority of 
existing SDP attributes on the premise that they are of no interest to 
WEBRTC.

/a

On 2/2/13 18:13, Harald Alvestrand wrote:
> in your scheme 2, how would you represent multicast and 
> transport-mixer based sessions?
>
>
>> Sorry for the relatively short lead time before next week's meeting 
>> -- I had meant to put this kind of summary together a week or two 
>> ago, but have been far busier than I anticipated.
>>
>> /a

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 5 Feb 2013 16:00:50 +0000

Please see inline.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Adam Roach
> Sent: Tuesday, February 05, 2013 9:33 AM
> To: Justin Uberti
> Cc: [email-masked]; Peter Thatcher
> Subject: Re: [MMUSIC] Thoughts on m= lines and media streams
> 
> rejects the m= line that contains all the attributes (e.g. a=ice-ufrag,
> a=crypto, etc)
> 
> 
> Yeah, this part of the solution was put together kind of off-the-cuff. I'd
> considered three approaches, and this one was the least verbose. My
> anyway, is mandating that the transport-level attributes are in all the m=
> sections to which they apply. You end up with redundant information, but it
> does get around issues with rejecting some sections.

Why not specify such  transport attributes at the session level. For those attributes that are currently media level only, extend them to define how they work at the session level.

For example, b= at the session level is already defined to be the maximum bandwidth across the entire session whereas b= at the m=section level is for that m=section only.
For media direction/mode, a=sendonly/sendrecv/inactive/recvonly can be specified at the session level, in which case it applies to all m=sections unless they specify their own value.
Similarly, ICE candidates could be specified at the session level and have those apply at all media levels, unless the media level specifies its own value.

Cheers,
Charles

> together, and there's more than a little carelessness in the SDP examples.
> That shouldn't be there.
> 
> 
> 
> And this is the sloppiness I referred to above. The different sections should
> have non-overlapping RTP payload types.
> 
> 
> 
> Herring" to me. Practically speaking, I think there are few, if any, real-world
> ask for a low-bitrate codec for stream A and a high-bitrate codec for stream
> B (where the codecs used are different). A more realistic example would be
> where one might want a lower bitrate  for a specific multi-rate codec (e.g.
> Opus) to be used for stream A, and higher bitrate for stream B. This of
> course can be handled with the per-ssrc fmtp attributes that are specified in
> RFC 5576. However, I think in almost all cases, the receiver is in a much
> worse position to make these decisions that the sender; the sender, aware
> of the estimated bandwidth to the receiver, and the relative
> has little or no impact) is a better party to select the quality of a stream than
> the person receiving it?
> 
> I have nothing to say to that. It's sufficiently false on its face that I really
> have nowhere to start grappling with it.
> 
> 
> 
> important consequence of the fact that a=ssrc indications are unidirectional,
> namely that streams could be added or removed by both endpoints
> simultaneously without causing a glare condition. m= lines, on the other
> hand, since they are identified by their index, do not have this property; if
> both sides decide to add a m= line at the same time, bad things happen.
> 
> 
> None of the three approaches avoids the necessity to handle glare. With a
> stateful offer/answer approach, there's simply no way around it. We would
> have to go back to the drawing board if we wanted to come up with a
> system that avoided glare (and even then, I don't think success is very
> likely).
> 
> 
> 
> Aside from the one case of having video + presentation streams, I have not
> yet heard of widely deployed equipment that uses multiple audio or video
> m= lines, and certainly any such equipment will not understand BUNDLE or
> MMT. Therefore a gateway is required, and any gateway that was
> performing BUNDLE or MMT could certainly do the minor additional work to
> package the SDP as 1 m= line (remember, it already needs to add the a=ssrc
> attributes). Ergo, I don't agree that the general case of N m= lines needs to
> be handled by all RTCWEB implementations.
> 
> 
> Truth be told, even though I called this "legacy interop," it's more accurate
> to think of it as "interop with non-RTCWEB implementations that use SDP,"
> of which legacy is an important subset.
> 
> rewrites, then I question the use of SDP at all in RTCWEB. The selection of
> SDP, to my understanding, was to provide simpler interop with non-RTCWEB
> implementations. Once we assume that you're necessarily rewriting it for
> such interop to work, we may as well abandon it altogether.
> 
> /a

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 5 Feb 2013 16:08:42 +0000

Hi,

> Similarly, ICE candidates could be specified at the session level and have those apply at all media levels, unless the media level specifies its own value.

That has been considered, but how would it work if the remote endpoint doesn't support the mechanism?

Regards,

Christer

---

From: "Roni Even" <[email-masked]>
Date: Tue, 5 Feb 2013 18:14:27 +0200

Hi,
One requirement that may be important for your analysis is if there must be
one offer answer exchange or more for setting the call. 
Some of the current solutions are doing a second one in order to reflect the
current state and prevent codec switching or new content without explicit
signaling. This is because they are not capable of doing it. This can
address the SBC issue that expect to have the correct SDP.
BTW: this is also an issue in CLUE since the initial offer may have a
minimal set and the actual SDP can only be specified after the CLUE
negotitation.

Roni Even

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Adam Roach
Sent: 01 February, 2013 2:18 AM
To: [email-masked]
Subject: [MMUSIC] Thoughts on m= lines and media streams

Based on the conversations we've had so far, I've put together the following
summary of my thoughts on the topic of whether we want to put multiple media
streams in a single m= line.

Sorry for the relatively short lead time before next week's meeting -- I had
meant to put this kind of summary together a week or two ago, but have been
far busier than I anticipated.

/a

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Tue, 5 Feb 2013 17:05:25 +0000

> Behalf Of Christer Holmberg
> Sent: Tuesday, February 05, 2013 11:09 AM
> To: Charles Eckel (eckelcu); Adam Roach; Justin Uberti
> > Why not specify such transport attributes at the session level. For those
> attributes that are currently media level only, extend them to define how
> they work at the session level.
> >
> > For example, b= at the session level is already defined to be the maximum
> bandwidth across the entire session whereas b= at the m=section level is for
> that m=section only.
> > For media direction/mode, a=sendonly/sendrecv/inactive/recvonly can be
> specified at the session level, in which case it applies to all m=sections
> unless they specify their own value.
> > Similarly, ICE candidates could be specified at the session level and have
> those apply at all media levels, unless the media level specifies its own
> value.
> 

Crap. That said, I think you have that same concern with any approach. Using ICE candidates as an example, how do you know if the remote endpoint supports the same candidate existing in multiple m=sections and will use them as intended.

Cheers,
Charles

---

From: Justin Uberti <[email-masked]>
Date: Tue, 5 Feb 2013 09:17:49 -0800

On Tue, Feb 5, 2013 at 6:32 AM, Adam Roach <[email-masked]> wrote:

> considered three approaches, and this one was the least verbose. My second
> choice, which has better backwards compatibility properties anyway, is
> mandating that the transport-level attributes are in all the m= sections to
> which they apply. You end up with redundant information, but it does get
> around issues with rejecting some sections.
>
>
> demultiplexing the received streams. IOW, proposal 2 requires RFC 5576 just
> like 1a and 1b.
>
>

Alas, RTP payload types are not sufficient for demux, since RTCP packets
don't include RTP PTs.

> mechanism to ask for a low-bitrate codec for stream A and a high-bitrate
> codec for stream B (where the codecs used are different). A more realistic
> example would be where one might want a lower bitrate  for a specific
> multi-rate codec (e.g. Opus) to be used for stream A, and higher bitrate
> for stream B. This of course can be handled with the per-ssrc fmtp
> attributes that are specified in RFC 5576. However, I think in almost all
> cases, the receiver is in a much worse position to make these decisions
> that the sender; the sender, aware of the estimated bandwidth to the
> receiver, and the relative value/complexity of its streams, can make more
> informed decisions about bandwidth allocation, and perform them without any
> signaling delay.
>
>
> I have nothing to say to that. It's sufficiently false on its face that I
> really have nowhere to start grappling with it.
>

I should have been more clear. Yes, the receiver is the one who needs to
choose the relative priorities of streams, but the sender is in a better
place to make actual decisions about how those preferences should be
carried out (for example, what bitrate should be sent, or which layers of a
layered codec should be used).

Even if the receiver retains full control, the receiver could still ask for
different bitrates given using the mechanisms I described above.

> have to go back to the drawing board if we wanted to come up with a system
> that avoided glare (and even then, I don't think success is very likely).
>

Since WebRTC leaves the signaling up to the application, if your signaling
system allows for partial updates, you can have "stream-add" and
"stream-remove" operations that are glareless.

---

From: Adam Roach <[email-masked]>
Date: Tue, 05 Feb 2013 12:30:25 -0500

On 2/5/13 12:17, Justin Uberti wrote:
> Even if the receiver retains full control, the receiver could still 
> ask for different bitrates given using the mechanisms I described above.

Ah, yes, I forgot to speak to those directly. You're making assumptions 
here that all the codecs we want to use will be one-size-fits-all (with 
all adjustments to all uses possible by parameters on those codecs) from 
now until this technology is obsolete. I think that's short-sighted.

/a

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 05 Feb 2013 19:54:54 +0100

> existing SDP attributes on the premise that they are of no interest to

Told you I hated to do this :-)