Subject: Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

From: Bo Burman <[email-masked]>
Date: Mon, 2 Dec 2013 10:52:09 +0000

Should a simulcast-capable endpoint be restricted to be a simulcast-sender-only or a simulcast-receiver-only, or do we envision endpoints that are capable of both sending and receiving simulcast? One example would be if simulcast streams are to be forwarded between different SFU (using Bernard's terminology below).

If so and assuming that we define a new 5888-type simulcast grouping, do we need separate groupings for send direction simulcast and receive direction simulcast, or is a=sendonly and a=recvonly information for simulcast-grouped media blocks fully sufficient?

Also, I think the way to use this approach for simulcast in combination with multiple separate media sources (like multiple cameras) or media sinks (like multiple displays) would be pretty straightforward; just add one simulcast grouping per media source/sink.

From: mmusic [mailto:[email-masked]] On Behalf Of Bernard Aboba
Sent: den 30 november 2013 20:31
To: Christer Holmberg; [email-masked]
Subject: Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

Christer asked:

"In your example, how does the answerer know that the offered m- lines are for simulcast? As far as O/A is concerned, it could just be a number of completely separate (different content etc) uni-directional streams."

[BA] The most obvious way would be grouping.  Looking at RFC 5888/5583, it doesn't appear that either FID or DDP grouping would apply to simulcast.   So perhaps a new group type would be needed.

Bernard Aboba <[email-masked]<mailto:[email-masked]>> wrote:

At IETF 88, a number of  participants (including myself) expressed the view that draft-westerlund-avtcore-rtp-simulcast was too complicated.  Below, I will attempt to explain how simulcast might be negotiated in a simplified Offer/Answer exchange based on the principles of Unified Plan.

Typically in simulcast we have an endpoint sending multiple streams in parallel to the Selective Forwarding Unit (SFU), while receiving one among a set of potential streams from the SFU, with the SFU being able to switch between the streams that it is sending to the endpoint without signaling.  RFC 3264 describes how to handle each of these situations.

With respect to sending multiple streams in parallel, RFC 3264 Section 5.1 states the following:

If the offerer wishes to only send media on a stream to its peer, it MUST mark the stream as sendonly with the "a=sendonly" attribute.  We  refer to a stream as being marked with a certain direction if a direction attribute was present as either a media stream attribute or a session attribute. For sendonly RTP streams, the payload type numbers indicate the value of the payload type field in RTP packet the offerer is planning to send for that codec...  In all cases, the formats in the "m=" line MUST be listed in order of preference, with the first format listed being preferred.  In this case, preferred means that the recipient of the offer SHOULD use the format with the highest preference that is acceptable to it. If multiple media streams of different types are present, it means that the offerer wishes to use those streams at the same time.  A typical case is an audio and a video stream as part of a videoconference.

If multiple media streams of the same type are present in an offer,  it means that the offerer wishes to send (and/or receive) multiple streams of that type at the same time...  Each stream MAY use different encodings.

With respect to receiving one among a set of potential streams, here is what RFC 3264 Section 5.1 says:

If the offerer wishes to only receive media from its peer, it MUST mark the stream as recvonly... If multiple formats are listed, it means that the offerer is capable of making use of any of those formats during the session.  In other words, the answerer MAY change formats in the middle of the session, making use of any of the formats listed, without sending a new offer.  For a sendonly stream, the offer SHOULD indicate those formats the offerer is willing to send for this stream.  For a recvonly stream,  the offer SHOULD indicate those formats the offerer is willing to receive for this stream....

For recvonly RTP streams, the payload type numbers indicate the value of the payload type field in RTP packets the offerer is expecting to  receive for that codec.

Based on the above:

1. An offer from an endpoint to an SFU would include an m=line for each potential simulcast stream to be sent, each including an a=sendonly line.  The offer would also include a single m=line (with a=recvonly) describing the streams that the endpoint is prepared to receive, each with their own payload type.

2. An answer from an SFU to an endpoint would accept or reject each of the simulcast streams that the endpoint is proposing to send (with an a=recvonly line).  In addition, the answer would include a single m=line (with a=sendonly) indicating the streams from among those proposed by the endpoint that the SFU is prepared to send.  The SFU can then switch between those streams without having to signal the endpoint.

Since it is possible for the Answerer to accept/reject each of the proposed streams the Offerer is proposing to send, an Answerer can indicate how many streams it is prepared to receive (might only be one if simulcast isn't supported).   Similarly, an Answerer can select those simulcast streams it is prepared to send (might only be one if simulcast isn't supported).

Example Offer (leaving out BUNDLE, rtp/rtcp multiplexing, and audio):

   v=0
   o=alice 2362969037 2362969040 IN IP4 192.0.2.156
   s=Simulcast Enabled Unified Plan Client
   t=0 0
   c=IN IP4 192.0.2.156
   b=AS:665
   m=video 49300 RTP/AVP 97
   a=sendonly
   b=AS:520
   a=rtpmap:97 H264/90000
   a=fmtp:97 profile-level-id=42c01e
   a=imageattr:97 send [x=640,y=360]
   m=video 49302 RTP/AVP 98
   a=sendonly
   a=rtpmap:98 H264/90000
   a=fmtp:98 profile-level-id=42c01e
   a=imageattr:98 send [x=320,y=180]
   m=video 49304 RTP/AVP 100 99
   a=recvonly
   a=rtpmap:100 H264/90000
   a=fmtp:100 profile-level-id=42c01e
   a=imageattr:100 recv [x=640,y=360]
   a=rtpmap:99 H264/90000
   a=fmtp:99 profile-level-id=42c01e
   a=imageattr:99 recv [x=320,y=180]

Example Answer:

   v=0
   o=server 823479283 1209384938 IN IP4 192.0.2.2
   s=Answer to Simulcast Enabled Unified Plan Client
   t=0 0
   c=IN IP4 192.0.2.43
   b=AS:665
   m=video 48000 RTP/AVP 97
   a=recvonly
   b=AS:520
   a=rtpmap:97 H264/90000
   a=fmtp:97 profile-level-id=42c01e
   a=imageattr:97 recv [x=640,y=360]
   m=video 48002 RTP/AVP 98
   a=recvonly
   a=rtpmap:98 H264/90000
   a=fmtp:98 profile-level-id=42c01e
   a=imageattr:98 recv [x=320,y=180]
   m=video 48004 RTP/AVP 100 99
   a=sendonly
   a=rtpmap:100 H264/90000
   a=fmtp:100 profile-level-id=42c01e
   a=imageattr:100 send [x=640,y=360]
   a=rtpmap:99 H264/90000
   a=fmtp:99 profile-level-id=42c01e
   a=imageattr:100 send [x=320,y=180]

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 2 Dec 2013 11:12:59 +0000

Hi,

If the answerer does NOT support simulcast, it will see an Offer with multiple uni-directional streams. The behavior is difficult to predict.

So, even with a mechanism based on grouping, the question is still: should the support of the simulcast feature be negotiated between the endpoints first, before it is enabled? Most likely it would require 2 O/A transactions.

Regards,

Christer

From: Bo Burman
Sent: 2. joulukuuta 2013 12:52
To: Bernard Aboba; Christer Holmberg; [email-masked]
Subject: RE: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

From: mmusic [mailto:[email-masked]] On Behalf Of Bernard Aboba
Sent: den 30 november 2013 20:31
To: Christer Holmberg; [email-masked]<mailto:[email-masked]>
Subject: Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

Christer asked:

Based on the above:

   v=0

Example Answer:

   v=0

---

From: Bo Burman <[email-masked]>
Date: Mon, 2 Dec 2013 11:55:10 +0000

Agree. This approach with having the ability to negotiate "the simulcast feature" in a first O/A is already present in draft-westerlund-avtcore-rtp-simulcast. It is however not required to use 2 O/A, in case simulcast support is somehow known or at least very likely.

From: Christer Holmberg
Sent: den 2 december 2013 12:13
To: Bo Burman; Bernard Aboba; [email-masked]
Subject: RE: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

Hi,

Regards,

Christer

From: Bo Burman
Sent: 2. joulukuuta 2013 12:52
To: Bernard Aboba; Christer Holmberg; [email-masked]<mailto:[email-masked]>
Subject: RE: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast

Christer asked:

Based on the above:

   v=0

Example Answer:

   v=0

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2013 18:14:32 -0800

Bo Burman said: 
  
"Should a simulcast-capable endpoint be restricted to be a simulcast-sender-only or a simulcast-receiver-only, or do we envision endpoints that are capable of both sending and receiving simulcast? One example would be if simulcast streams are to be forwarded between different SFU (using Bernard’s terminology below).

Also, I think the way to use this approach for simulcast in combination with multiple separate media sources (like multiple cameras) or media sinks (like multiple displays) would be pretty straightforward; just add one simulcast grouping per media source/sink."
 
[BA] In the case where parallel simulcast streams are being both sent and received (e.g. between SFUs), there wouldn't be a need for a=sendonly or a=recvonly lines;  each m=line would apply to both sending and receiving.  So it seems like a single simulcast group would suffice.   If you have multiple sources or sinks, then yes, I believe you would have one simulcast grouping per media source/sink. A simple SFU1-SFU2 exchange would look like this: 
 
Offer: 
 
v=0
o=SFU1 2362969037 2362969040 IN IP4 192.0.2.156
s=Simulcast Enabled SFU1
t=0 0
c=IN IP4 192.0.2.156
b=AS:665
a=group:SC S1 S2
m=video 49300 RTP/AVP 97
b=AS:520
a=rtpmap:97 H264/90000
a=mid:S1
a=mid:S2
 
Answer: 
 
v=0
o=SFU2 823479283 1209384938 IN IP4 192.0.2.2
s=Answer to Simulcast Enabled SFU1
t=0 0
c=IN IP4 192.0.2.43
b=AS:665
a=group:SC S1 S2
m=video 48000 RTP/AVP 97
b=AS:520
a=rtpmap:97 H264/90000
a=fmtp:97 profile-level-id=42c01e
a=imageattr:97 recv [x=640,y=360]
a=mid:S1 
m=video 48002 RTP/AVP 98
a=rtpmap:98 H264/90000
a=fmtp:98 profile-level-id=42c01e
a=imageattr:98 recv [x=320,y=180]
a=mid:S2

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2013 18:17:17 -0800

Christer said: 
 
"If the answerer does NOT support simulcast, it will see an Offer with multiple uni-directional streams. The behavior is difficult to predict. So, even with a mechanism based on grouping, the question is still: should the support of the simulcast feature be negotiated between the endpoints first, before it is enabled? Most likely it would require 2 O/A transactions."
 
[BA] If the answerer does NOT support simulcast, it would not respond with an a=group:SC line, so the Offerer will know that simulcast is not supported.  If the Answerer accepted more than one of the m= lines offered by the Offerer, but didn't include an a=group:SC line, then the Offerer could send a modified offer in order to avoid confusion on the part of the Answerer.   A similar scenario can occur in draft-westerlund-avtcore-rtp-simulcast, since the Answerer may not understand the a=sim-send-cap or a=sim-recv-cap lines.  
 
In an O/A exchange where both the Offerer and Answerer support simulcast, the exchange would look like this: 
 
Offer: 
 
v=0
s=Simulcast Enabled Client
a=mid:S3
 
Answer: 
 
v=0
s=Answer to Simulcast Enabled Client
a=imageattr:99 send [x=320,y=180]
a=mid:S3

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 3 Dec 2013 09:55:51 +0000

Hi,

>Christer said:
>
>"If the answerer does NOT support simulcast, it will see an Offer with multiple uni-directional streams. The behavior is difficult to predict.
>
>So, even with a mechanism based on grouping, the question is still: should the support of the simulcast feature be negotiated between the endpoints first, before it is enabled? Most likely it would require 2 O/A transactions."
>
>[BA] If the answerer does NOT support simulcast, it would not respond with an a=group:SC line, so the Offerer will know that simulcast is not supported.  If the Answerer accepted more than one of the m= lines offered by the Offerer, but >didn't include an a=group:SC line, then the Offerer could send a modified offer in order to avoid confusion on the part of the Answerer.   A similar scenario can occur in draft-westerlund-avtcore-rtp-simulcast, since the Answerer may not >understand the a=sim-send-cap or a=sim-recv-cap lines.

Sure. My comment was on the usage of the bundle mechanism in general, not a comparison with draft-westerlund :)

Regards,

Christer

Offer:
v=0

From: Bo Burman

Christer asked:

Based on the above:

   v=0

Example Answer:

   v=0

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2013 16:05:34 -0800

Christer said: 

So, even with a mechanism based on grouping, the question is still: should the support of the simulcast feature be negotiated between the endpoints first, before
 it is enabled? Most likely it would require 2 O/A transactions." [BA] If the answerer does NOT support simulcast, it would not respond with an a=group:SC line, so the Offerer will know that simulcast is not supported.  If the Answerer accepted more than one of the m= lines offered by the Offerer, but didn't include an a=group:SC line, then the Offerer could send a modified offer in order to avoid confusion on the part of the Answerer.   A similar scenario can occur in draft-westerlund-avtcore-rtp-simulcast, since the Answerer may not understand the a=sim-send-cap or a=sim-recv-cap lines.  In an O/A exchange where both the Offerer and Answerer support simulcast, the exchange would look like this:  Offer: v=0
o=alice 2362969037 2362969040 IN IP4 192.0.2.156
s=Simulcast Enabled Client
t=0 0
c=IN IP4 192.0.2.156
b=AS:665a=group:SC S1 S2
m=video 49300 RTP/AVP 97
a=sendonly
b=AS:520
a=rtpmap:97 H264/90000a=fmtp:97 profile-level-id=42c01e
a=imageattr:97 send [x=640,y=360]a=mid:S1
m=video 49302 RTP/AVP 98
a=sendonly
a=rtpmap:98 H264/90000
a=fmtp:98 profile-level-id=42c01e
a=imageattr:98 send [x=320,y=180]a=mid:S2
m=video 49304 RTP/AVP 100 99
a=recvonly
a=rtpmap:100 H264/90000
a=fmtp:100 profile-level-id=42c01e
a=imageattr:100 recv [x=640,y=360] 
a=rtpmap:99 H264/90000
a=fmtp:99 profile-level-id=42c01e
a=imageattr:99 recv [x=320,y=180]a=mid:S3Answer:  v=0
o=server 823479283 1209384938 IN IP4 192.0.2.2
s=Answer to Simulcast Enabled Clientt=0 0
c=IN IP4 192.0.2.43b=AS:665a=group:SC S1 S2
m=video 48000 RTP/AVP 97
a=recvonly
b=AS:520a=rtpmap:97 H264/90000a=fmtp:97 profile-level-id=42c01e
a=imageattr:97 recv [x=640,y=360]a=mid:S1 m=video 48002 RTP/AVP 98
a=recvonly
a=rtpmap:98 H264/90000a=fmtp:98 profile-level-id=42c01e
a=imageattr:98 recv [x=320,y=180]a=mid:S2
m=video 48004 RTP/AVP 100 99a=sendonly
a=rtpmap:100 H264/90000
a=fmtp:100 profile-level-id=42c01e
a=imageattr:100 send [x=640,y=360] 
a=rtpmap:99 H264/90000
a=fmtp:99 profile-level-id=42c01e
a=imageattr:99 send [x=320,y=180]a=mid:S3  

From: Bo Burman

 
Should a simulcast-capable endpoint be restricted to be a simulcast-sender-only or a simulcast-receiver-only, or do we envision endpoints that are capable of
 both sending and receiving simulcast? One example would be if simulcast streams are to be forwarded between different SFU (using Bernard’s terminology below).
 
If so and assuming that we define a new 5888-type simulcast grouping, do we need separate groupings for send direction simulcast and receive direction simulcast,
 or is a=sendonly and a=recvonly information for simulcast-grouped media blocks fully sufficient?
 
Also, I think the way to use this approach for simulcast in combination with multiple separate media sources (like multiple cameras) or media sinks (like multiple
 displays) would be pretty straightforward; just add one simulcast grouping per media source/sink.

From: mmusic [mailto:[email-masked]]
On Behalf Of Bernard Aboba

 

Christer asked:

Bernard Aboba <[email-masked]> wrote:

At IETF 88, a number of  participants (including myself) expressed the view that draft-westerlund-avtcore-rtp-simulcast was too complicated.  Below, I will attempt to explain how simulcast
 might be negotiated in a simplified Offer/Answer exchange based on the principles of Unified Plan.

Typically in simulcast we have an endpoint sending multiple streams in parallel to the Selective Forwarding Unit (SFU), while receiving one among a set of potential streams from the SFU, with the SFU being able to switch between the streams that it is sending
 to the endpoint without signaling.  RFC 3264 describes how to handle each of these situations.

 
If the offerer wishes to only send media on a stream to its peer, it MUST mark the stream as sendonly with the "a=sendonly" attribute.  We  refer to a stream as being marked with a certain
 direction if a direction attribute was present as either a media stream attribute or a session attribute. For sendonly RTP streams, the payload type numbers indicate the value of the payload type field in RTP packet the offerer is planning to send for that
 codec...  In all cases, the formats in the "m=" line MUST be listed in order of preference, with the first format listed being preferred.  In this case, preferred means that the recipient of the offer SHOULD use the format with the highest preference that
 is acceptable to it. If multiple media streams of different types are present, it means that the offerer wishes to use those streams at the same time.  A typical case is an audio and a video stream as part of a videoconference.

 
If the offerer wishes to only receive media from its peer, it MUST mark the stream as recvonly... If multiple formats are listed, it means that the offerer is capable of making use of any
 of those formats during the session.  In other words, the answerer MAY change formats in the middle of the session, making use of any of the formats listed, without sending a new offer.  For a sendonly stream, the offer SHOULD indicate those formats the offerer
 is willing to send for this stream.  For a recvonly stream,  the offer SHOULD indicate those formats the offerer is willing to receive for this stream....

Based on the above:

1. An offer from an endpoint to an SFU would include an m=line for each potential simulcast stream to be sent, each including an a=sendonly line.  The offer would also include a single m=line (with a=recvonly) describing the streams that the endpoint is prepared
 to receive, each with their own payload type. 

2. An answer from an SFU to an endpoint would accept or reject each of the simulcast streams that the endpoint is proposing to send (with an a=recvonly line).  In addition, the answer would include a single m=line (with a=sendonly) indicating the streams from
 among those proposed by the endpoint that the SFU is prepared to send.  The SFU can then switch between those streams without having to signal the endpoint.

Since it is possible for the Answerer to accept/reject each of the proposed streams the Offerer is proposing to send, an Answerer can indicate how many streams it is prepared to receive (might only be one if simulcast isn't supported).   Similarly, an Answerer
 can select those simulcast streams it is prepared to send (might only be one if simulcast isn't supported).

   v=0

   t=0 0

   c=IN IP4 192.0.2.156

   b=AS:665

   m=video 49300 RTP/AVP 97

   a=sendonly

   b=AS:520

   a=rtpmap:97 H264/90000

   m=video 49302 RTP/AVP 98

   a=sendonly

   a=rtpmap:98 H264/90000

   a=recvonly

   a=rtpmap:100 H264/90000

   a=rtpmap:99 H264/90000

 

Example Answer: 

   v=0

   t=0 0

   c=IN IP4 192.0.2.43

   b=AS:665

   m=video 48000 RTP/AVP 97

   a=recvonly

   b=AS:520

   a=rtpmap:97 H264/90000

   m=video 48002 RTP/AVP 98

   a=recvonly

   a=rtpmap:98 H264/90000

   a=sendonly

   a=rtpmap:100 H264/90000

   a=rtpmap:99 H264/90000

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 2 Dec 2013 16:15:20 -0800

Bo Burman said: 

"Should a simulcast-capable endpoint be restricted to be a simulcast-sender-only or a simulcast-receiver-only, or do we envision endpoints that are capable of
 both sending and receiving simulcast? One example would be if simulcast streams are to be forwarded between different SFU (using Bernard’s terminology below)." 
If so and assuming that we define a new 5888-type simulcast grouping, do we need separate groupings for send direction simulcast and receive direction simulcast,
 or is a=sendonly and a=recvonly information for simulcast-grouped media blocks fully sufficient? Also, I think the way to use this approach for simulcast in combination with multiple separate media sources (like multiple cameras) or media sinks (like multiple displays) would be pretty straightforward; just add one simulcast grouping per media source/sink." [BA] In the case where parallel simulcast streams are being both sent and received (e.g. between SFUs), there wouldn't be a need for a=sendonly or a=recvonly lines;  each m=line would apply to both sending and receiving.  So it seems like a single simulcast group would suffice.   If you have multiple sources or sinks, then yes, I believe you would have one simulcast grouping per media source/sink.  A simple SFU1-SFU2 exchange would look like this:  Offer: 

v=0

o=SFU1
2362969037
2362969040 IN IP4 192.0.2.156

s=Simulcast
Enabled
SFU1t=0
0

c=IN
IP4
192.0.2.156

b=AS:665a=group:SC S1 S2

m=video
49300
RTP/AVP 97

b=AS:520

a=rtpmap:97
H264/90000a=fmtp:97
profile-level-id=42c01e

a=imageattr:97
send
[x=640,y=360]a=mid:S1

m=video
49302
RTP/AVP 98

a=rtpmap:98
H264/90000

a=fmtp:98
profile-level-id=42c01e

a=imageattr:98
send
[x=320,y=180]a=mid:S2

Answer: 

v=0

o=SFU2
823479283
1209384938 IN IP4 192.0.2.2

s=Answer
to
Simulcast Enabled
SFU1t=0 0

c=IN
IP4
192.0.2.43b=AS:665a=group:SC S1
S2

m=video
48000
RTP/AVP 97

b=AS:520a=rtpmap:97
H264/90000a=fmtp:97
profile-level-id=42c01e

a=imageattr:97
recv
[x=640,y=360]a=mid:S1
m=video
48002
RTP/AVP 98

a=rtpmap:98
H264/90000a=fmtp:98
profile-level-id=42c01e

a=imageattr:98
recv
[x=320,y=180]a=mid:S2

 

Christer asked:

 

 

 

 

Based on the above:

 

 

   v=0

   t=0 0

   c=IN IP4 192.0.2.156

   b=AS:665

   m=video 49300 RTP/AVP 97

   a=sendonly

   b=AS:520

   a=rtpmap:97 H264/90000

   m=video 49302 RTP/AVP 98

   a=sendonly

   a=rtpmap:98 H264/90000

   a=recvonly

   a=rtpmap:100 H264/90000

   a=rtpmap:99 H264/90000

 

Example Answer: 

   v=0

   t=0 0

   c=IN IP4 192.0.2.43

   b=AS:665

   m=video 48000 RTP/AVP 97

   a=recvonly

   b=AS:520

   a=rtpmap:97 H264/90000

   m=video 48002 RTP/AVP 98

   a=recvonly

   a=rtpmap:98 H264/90000

   a=sendonly

   a=rtpmap:100 H264/90000

   a=rtpmap:99 H264/90000

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 03 Dec 2013 12:55:16 -0500

ISTM there are major asymmetries here. Whether something can be 
simulcast can only be determined by the sender.

- A sender, when making an offer, can include multiple sendonly streams 
marked as simulcasts of each other.

- A receiver, when presented with an offer containing simulcast, can 
decide which of the simulcast streams it wants to receive and describe 
them as recvonly.

- A receiver making an offer could indicate that it is interested in 
receiving simulcast versions of the same source, but only in special 
cases would it know that it makes sense to send such an offer. Using 
grouping, there could be an ambiguity whether the goal is simply to 
receive *one* of the alternative formats, or if the goal is to receive 
multiple simulcast streams. (Maybe treating them as alternatives can be 
banned because other techniques should be used to indicate that.)

- A sender, when presented with an offer containing simulcast recvonly 
streams can decide if it is capable of satisfying more than one of those.

I *think* the sendrecv case can be treated as just a combination with no 
new semantics. The answerer would analyze the offer once as a simulcast 
receiver and once as a simulcast sender. This could result in arbitrary 
combinations of sendrecv, sendonly, and recvonly in the answer for the 
m-lines in the simulcast group.

	Thanks,
	Paul

On 12/2/13 7:15 PM, Bernard Aboba wrote:
> Bo Burman said:
>
> "Should a simulcast-capable endpoint be restricted to be a
> simulcast-sender-only or a simulcast-receiver-only, or do we envision
> endpoints that are capable of both sending and receiving simulcast? One
> example would be if simulcast streams are to be forwarded between
> different SFU (using Bernard’s terminology below)."
>
> If so and assuming that we define a new 5888-type simulcast grouping, do
> we need separate groupings for send direction simulcast and receive
> direction simulcast, or is a=sendonly and a=recvonly information for
> simulcast-grouped media blocks fully sufficient?
>
> Also, I think the way to use this approach for simulcast in combination
> with multiple separate media sources (like multiple cameras) or media
> sinks (like multiple displays) would be pretty straightforward; just add
> one simulcast grouping per media source/sink."
>
> [BA] In the case where parallel simulcast streams are being both sent
> and received (e.g. between SFUs), there wouldn't be a need for
> a=sendonly or a=recvonly lines;  each m=line would apply to both sending
> and receiving.  So it seems like a single simulcast group would
> suffice.   If you have multiple sources or sinks, then yes, I believe
> you would have one simulcast grouping per media source/sink.
>
> A simple SFU1-SFU2 exchange would look like this:
>
> Offer:
>
> v=0
> o=SFU12362969037 2362969040 IN IP4 192.0.2.156
> s=Simulcast Enabled SFU1
>
> t=0 0
> c=IN IP4 192.0.2.156
> b=AS:665
>
> a=group:SCS1 S2
> m=video 49300 RTP/AVP 97
> b=AS:520
> a=rtpmap:97 H264/90000
>
> a=group:SCS1 S2
> a=imageattr:97 recv[x=640,y=360]
>
> a=mid:S1
>
> m=video 48002 RTP/AVP 98
> a=rtpmap:98 H264/90000
>
> a=imageattr:98 recv[x=320,y=180]
>
> a=mid:S2
>
> *From:*mmusic [mailto:[email-masked]] *On Behalf Of *Bernard Aboba
> *Sent:* den 30 november 2013 20:31
> *To:* Christer Holmberg; [email-masked]
> *Subject:* Re: [MMUSIC] Simplifying draft-westerlund-avtcore-rtp-simulcast
>
> Christer asked:
>
> Bernard Aboba <[email-masked]  <mailto:[email-masked]>> wrote:
>
>
>
> At IETF 88, a number of  participants (including myself) expressed the
> view that draft-westerlund-avtcore-rtp-simulcast was too
> complicated.  Below, I will attempt to explain how simulcast might
> be negotiated in a simplified Offer/Answer exchange based on the
> principles of Unified Plan.
>
> Typically in simulcast we have an endpoint sending multiple streams in
> parallel to the Selective Forwarding Unit (SFU), while receiving one
> among a set of potential streams from the SFU, with the SFU being able
> to switch between the streams that it is sending to the endpoint without
> signaling.  RFC 3264 describes how to handle each of these situations.
>
> If the offerer wishes to only send media on a stream to its peer,
> it MUST mark the stream as sendonly with the "a=sendonly" attribute.
> We  refer to a stream as being marked with a certain direction if
> a direction attribute was present as either a media stream attribute or
> a session attribute. For sendonly RTP streams, the payload type numbers
> indicate the value of the payload type field in RTP packet the offerer
> is planning to send for that codec...  In all cases, the formats in the
> "m=" line MUST be listed in order of preference, with the first format
> listed being preferred.  In this case, preferred means that the
> recipient of the offer SHOULD use the format with the highest preference
> that is acceptable to it. If multiple media streams of different types
> are present, it means that the offerer wishes to use those streams at
> the same time.  A typical case is an audio and a video stream as part of
> a videoconference.
>
> If multiple media streams of the same type are present in an offer,  it
> means that the offerer wishes to send (and/or receive) multiple streams
> of that type at the same time...  Each stream MAY use different encodings.
>
>
> If the offerer wishes to only receive media from its peer, it MUST mark
> the stream as recvonly... If multiple formats are listed, it means that
> the offerer is capable of making use of any of those formats during the
> session.  In other words, the answerer MAY change formats in the middle
> of the session, making use of any of the formats listed, without sending
> a new offer.  For a sendonly stream, the offer SHOULD indicate those
> formats the offerer is willing to send for this stream.  For a recvonly
> stream,  the offer SHOULD indicate those formats the offerer is willing
> to receive for this stream....
>
> For recvonly RTP streams, the payload type numbers indicate the value of
> the payload type field in RTP packets the offerer is expecting to
> receive for that codec.
>
>
> Based on the above:
>
> 1. An offer from an endpoint to an SFU would include an m=line for each
> potential simulcast stream to be sent, each including an a=sendonly
> line.  The offer would also include a single m=line (with a=recvonly)
> describing the streams that the endpoint is prepared to receive, each
> with their own payload type.
>
> 2. An answer from an SFU to an endpoint would accept or reject each of
> the simulcast streams that the endpoint is proposing to send (with an
> a=recvonly line).  In addition, the answer would include a single m=line
> (with a=sendonly) indicating the streams from among those proposed by
> the endpoint that the SFU is prepared to send.  The SFU can then switch
> between those streams without having to signal the endpoint.
>
> Since it is possible for the Answerer to accept/reject each of the
> proposed streams the Offerer is proposing to send, an Answerer can
> indicate how many streams it is prepared to receive (might only be one
> if simulcast isn't supported).   Similarly, an Answerer can select those
> simulcast streams it is prepared to send (might only be one if simulcast
> isn't supported).
>
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 3 Dec 2013 10:34:19 -0800

Paul said:

"I *think* the sendrecv case can be treated as just a combination with no
new semantics. The answerer would analyze the offer once as a simulcast
receiver and once as a simulcast sender. This could result in arbitrary
combinations of sendrecv, sendonly, and recvonly in the answer for the
m-lines in the simulcast group."

[BA] Yes.  For this SFU-SFU case, the sendrecv m-lines in the Offer that
represent different representations from the same source would be included
on the a=group:SC line.  The Answerer could then decide how many streams it
wants to send and receive and mark the streams in the Answer appropriately.

Paul also said: 

"- A receiver making an offer could indicate that it is interested in
receiving simulcast versions of the same source, but only in special cases
would it know that it makes sense to send such an offer." 

[BA] Typically a client wouldn't indicate the desire to receive multiple
versions of the same source in parallel, that would be something that an SFU
would Offer (to a client or another SFU). 

Paul also said: 

"Using grouping, there could be an ambiguity whether the goal is simply to
receive *one* of the alternative formats, or if the goal is to receive
multiple simulcast streams. (Maybe treating them as alternatives can be
banned because other techniques should be used to indicate that.)"

[BA] If the goal is to receive *one* of the alternative formats, then the
alternatives are included on a single m-line (with a=recvonly) as per RFC
3264.   If the goal is to receive alternative formats in parallel, then
multiple m-lines would be used (again, as suggested in RFC 3264).  As to
what "mid" identifiers would be included on the a=group:SC line - to me,
"simulcast" means that multiple streams from a single source are being sent
(or received) in parallel.   So for the asymmetric example, I didn't include
the "mid" of the single m-line (with a=recvonly) on the a=group:SC line.    

Paul also said: 

"- A sender, when presented with an offer containing simulcast recvonly
streams can decide if it is capable of satisfying more than one of those."

[BA] If the Offer contains multiple m-lines marked "recvonly" that represent
different streams from the same source to be sent in parallel, this means
that the Offerer wants to receive simulcast, and so those m-lines would be
included in a simulcast group.  The Answerer could then decide if it wanted
to send more than one stream. 

	Thanks,
	Paul

On 12/2/13 7:15 PM, Bernard Aboba wrote:
> endpoints that are capable of both sending and receiving simulcast? 
> One example would be if simulcast streams are to be forwarded between 
> different SFU (using Bernard's terminology below)."
>
> If so and assuming that we define a new 5888-type simulcast grouping, 
> do we need separate groupings for send direction simulcast and receive 
> Also, I think the way to use this approach for simulcast in 
> combination with multiple separate media sources (like multiple 
> cameras) or media sinks (like multiple displays) would be pretty 
> straightforward; just add one simulcast grouping per media source/sink."
>
> a=sendonly or a=recvonly lines;  each m=line would apply to both 
> sending and receiving.  So it seems like a single simulcast group would
> o=SFU12362969037 2362969040 IN IP4 192.0.2.156 s=Simulcast Enabled 
> SFU1
>
> t=0 0
> c=IN IP4 192.0.2.156
> b=AS:665
>
> a=group:SCS1 S2
> m=video 49300 RTP/AVP 97
> b=AS:520
> a=rtpmap:97 H264/90000
>
> o=SFU2 823479283 1209384938 IN IP4 192.0.2.2 s=Answer to Simulcast 
> Enabled SFU1
>
> t=0 0
> c=IN IP4 192.0.2.43
>
> b=AS:665
>
> a=group:SCS1 S2
> m=video 48000 RTP/AVP 97
> b=AS:520
>
> a=rtpmap:97 H264/90000
>
> "In your example, how does the answerer know that the offered m- lines are
for simulcast? As far as O/A is concerned, it could just be a number of
completely separate (different content etc) uni-directional streams."
>
> [BA] The most obvious way would be grouping.  Looking at RFC 5888/5583, it
doesn't appear that either FID or DDP grouping would apply to simulcast.
So perhaps a new group type would be needed.
>
>
>
>
>
> Bernard Aboba <[email-masked]
<mailto:[email-masked]>> wrote:
> view that draft-westerlund-avtcore-rtp-simulcast was too complicated.  
> Below, I will attempt to explain how simulcast might be negotiated in 
> a simplified Offer/Answer exchange based on the principles of Unified 
> Plan.
>
> to switch between the streams that it is sending to the endpoint 
> without signaling.  RFC 3264 describes how to handle each of these
situations.
> If the offerer wishes to only send media on a stream to its peer, it 
> MUST mark the stream as sendonly with the "a=sendonly" attribute.
> We  refer to a stream as being marked with a certain direction if a 
> direction attribute was present as either a media stream attribute or 
> a session attribute. For sendonly RTP streams, the payload type 
> numbers indicate the value of the payload type field in RTP packet the 
> offerer is planning to send for that codec...  In all cases, the 
> formats in the "m=" line MUST be listed in order of preference, with 
> the first format listed being preferred.  In this case, preferred 
> means that the recipient of the offer SHOULD use the format with the 
> highest preference that is acceptable to it. If multiple media streams 
> of different types are present, it means that the offerer wishes to 
> use those streams at the same time.  A typical case is an audio and a 
> video stream as part of a videoconference.
>
> If multiple media streams of the same type are present in an offer,  
> it means that the offerer wishes to send (and/or receive) multiple 
> streams of that type at the same time...  Each stream MAY use different
encodings.
> If the offerer wishes to only receive media from its peer, it MUST 
> mark the stream as recvonly... If multiple formats are listed, it 
> means that the offerer is capable of making use of any of those 
> formats during the session.  In other words, the answerer MAY change 
> formats in the middle of the session, making use of any of the formats 
> listed, without sending a new offer.  For a sendonly stream, the offer 
> SHOULD indicate those formats the offerer is willing to send for this 
> stream.  For a recvonly stream,  the offer SHOULD indicate those 
> formats the offerer is willing to receive for this stream....
>
> For recvonly RTP streams, the payload type numbers indicate the value 
> of the payload type field in RTP packets the offerer is expecting to 
> receive for that codec.
>
>
> Based on the above:
>
> 1. An offer from an endpoint to an SFU would include an m=line for 
> each potential simulcast stream to be sent, each including an 
> a=sendonly line.  The offer would also include a single m=line (with 
> a=recvonly) describing the streams that the endpoint is prepared to 
> receive, each with their own payload type.
>
> a=recvonly line).  In addition, the answer would include a single 
> m=line (with a=sendonly) indicating the streams from among those 
> proposed by the endpoint that the SFU is prepared to send.  The SFU 
> can then switch between those streams without having to signal the
endpoint.
> simulcast streams it is prepared to send (might only be one if 
> simulcast isn't supported).
>