Subject: Re: [MMUSIC] [rtcweb] Translating Plan A into No Plan (Was: No Plan)

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 14:06:00 -0600

On Jun 3, 2013, at 11:28 AM, Emil Ivov <[email-masked]> wrote:

> An offer generated by a "No Plan" browser in this case would look something like this:
> 
>   v=0
>   o=- 20518 0 IN IP4 198.51.100.1
>   s=
>   t=0 0
>   c=IN IP4 203.0.113.1
>   a=ice-ufrag:F7gI
>   a=ice-pwd:x9cml/YzichV2+XlhiMu8g
>   a=fingerprint:sha-1 42:89:c5:c6:55:9d:39:f9:b6:eb:e7
>   a=group:BUNDLE m0 m1
> 
>   m=audio 56600 RTP/SAVPF 96 0
>   a=mid:m0
>   a=rtpmap:96 opus/48000
>   a=rtpmap:0 PCMU/8000
>   a=ptime:20
>   a=sendrecv
>   a=rtcp-mux
>   [ICE candidates]
> 
>   m=video 56602 RTP/SAVPF 97 98
>   a=mid:m1
>   a=rtpmap:97 H264/90000
>   a=fmtp:97 profile-level-id=4d0028;packetization-mode=1
>   a=rtpmap:98 VP8/90000
>   a=sendrecv
>   a=rtcp-mux
>   [ICE candidates]

In no-plan, What would the answer look like if you were receiving 2 video stream?

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:01:34 +0300

(moving from rtcweb)

Hey Eric,

On 03.06.13, 21:33, Eric Rescorla wrote:
>
> On Mon, Jun 3, 2013 at 10:28 AM, Emil Ivov <[email-masked]
> <mailto:[email-masked]>> wrote:
> [...]
>
>     III. Talking to other WebRTC applications
>
>     This is the fun case and the one we should be most concerned with.
>     Let's imagine that the answerer needs to add a fourth video stream.
>     To make this work endpoints would need to do the following:
>
>     a) with Plan A and draft-roach-rtcweb-glareless-__add:
>         - send application specific signalling to the offerer
>         - have a new O/A exchange
>
>     b) with Plan A:
>         - have a new O/A exchange
>         - potentially risk glare with some impact on user experience
>
>     c) with No Plan:
>         ... nothing
>
>
> I'm really not following how this works in "No Plan". Ignoring Plan A
> and Plan B, can you just
> walk through a complete example of what happens in your plan? I.e., how
> does browser
> B learn learn that Browser A wants to send 3 streams instead of one?

It just gets three streams instead of one. It sees three new SSRCs and 
it generates three new MediaStreams.

If these streams are supposed to reach the application and be rendered 
as separate tracks then this is all that needs to happen. They may need 
to be rendered in a specific order from left to right. But this only 
concerns the application.

There are of course cases where the browser does actually need to know 
about a relation between these three streams. This would be the case for 
various kinds of repair flows and layering.

Exactly how the browser would learn about these relations would depend 
on the specific mechanism and there are various options. They may be 
some that are handled entirely by the codec, or they could use something 
like the RTP header extension from the draft, or something like Magnus's 
SRCNAME.

They could even be specified in the SDP, but in that case they would be 
there only because the specific mechanism required it, not because 
RTCWEB did.

Does this make things clearer?

Cheers,
Emil

-- 
https://jitsi.org

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 3 Jun 2013 14:07:02 -0700

On Mon, Jun 3, 2013 at 2:01 PM, Emil Ivov <[email-masked]> wrote:

>>     a) with Plan A and draft-roach-rtcweb-glareless-_**_add:
>>
> It just gets three streams instead of one. It sees three new SSRCs and it
> generates three new MediaStreams.
>

So, just for clarity: you mean that A advertises one m-lin, B accepts,
and then A sends some arbitrary number of media flows, identified
by SSRC? If so, I've got some questions:

- If B can only display one video stream, how does it say so?
- How does B tell A not to send various streams?
- How does B tell A to send one stream big and another stream small?

-Ekr

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:29:38 +0300

Hey Eric,

On 04.06.13, 00:07, Eric Rescorla wrote:
>
> On Mon, Jun 3, 2013 at 2:01 PM, Emil Ivov <[email-masked]
>         <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>         [...]
>
>              a) with Plan A and draft-roach-rtcweb-glareless-____add:
>
>     It just gets three streams instead of one. It sees three new SSRCs
>     and it generates three new MediaStreams.
>
>
> and then A sends some arbitrary number of media flows,

Yes.

> identified
> by SSRC?

What does "identified" mean?

> If so, I've got some questions:
>

This is something that one app needs to tell another app. They can do so 
however they wish. As far as browsers are concerned: the transmitter 
just sends whatever the application instructs. The receiver just handles 
whatever it can.

> - How does B tell A not to send various streams?

What does "various" streams mean? If what you have in mind is: How does 
B tell A: "don't send me SSRCs 1, 2, 3 and 5" but keep 4 going, then B 
would send the list of SSRCs in app-specific signalling and A would stop 
sending them.

> - How does B tell A to send one stream big and another stream small?

Same as above.

Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Jun 2013 17:36:19 -0400

On 6/3/13 5:09 PM, Emil Ivov wrote:
> Hey Paul,
>
> On 03.06.13, 23:37, Paul Kyzivat wrote:
>> On 6/1/13 1:40 AM, Emil Ivov wrote:
>>>
>>>
>>> On 31.05.13, 22:11, Paul Kyzivat wrote:
>>>> On 5/31/13 6:51 AM, Emil Ivov wrote:
>>>>> Hey Paul,
>>>>>
>>>>> On 29.05.13, 22:57, Paul Kyzivat wrote:
>>>>>> Emil,
>>>>>>
>>>>>> I'm going to reserve judgment on this pending further discussion.
>>>>>> I think this *might* work for CLUE, but I want to be certain.
>>>>>
>>>>> Sure!
>>>>>
>>>>>> I'm concerned with decomposed endpoints that can't manage all the
>>>>>> streams on the same address/port. They will need multiple independent
>>>>>> m-lines and/or bundle groups.
>>>>>
>>>>> This is obviously open for debate but the general idea of No Plan is
>>>>> that: Offer/Answer is used for configuring media and RTP stacks and
>>>>> additional signalling is not the browser's concern.
>>>>>
>>>>> Having extra m= lines, particularly when using BUNDLE, is in many ways
>>>>> just extra signalling.
>>>>
>>>> You may be able to argue that adding extra m-lines into an existing
>>>> bundle is "just extra signaling". But introducing an additional 5-tuple
>>>> is more substantial. It requires configuring a new RTP stack and media.
>>>
>>> Indeed, this would require BUNDLE to work.
>>
>> I don't understand what that means - what point you are trying to make.
>
> Sorry about this. What I was trying to say was: yes, you are right,
> introducing a new 5-tuple is more substantial. Therefore adding a new m=
> line to an SDP is a lot simpler if it didn't also imply adding a new
> 5-tuple. This would only be possible when m= lines are bundled.

No. The problem arises without bundle. It gets more complex in 
combination with bundle.

> This is
> why I said: "Indeed, this would require BUNDLE to work."
>
> Is this clearer?

Maybe not. Still some disconnect here.

>> You seem to have some criterion for what requires sdp and what does not.
>> And AFAIK you are already assuming bundle works - you are simply using
>> it less.
>
> No, not really. The assumption is only necessary when trying to talk to
> Plan A devices and you want to have new m= lines added en route.

I thought you were assuming bundle what there to combine audio and video 
on a single 5-tuple. I don't think this need have anything to do with 
Plan A.

> No such assumption is necessary, for example, when talking to legacy SIP
> phones that will only show one video and one audio stream.

Of course, in such a degenerate case where there is only one m-line for 
each media type, and only one stream per m-line.

>> As best I can tell, you acknowledge that you need SDP to:
>> - negotiate a 5-tuple over which media will flow
>> - specify which types of media can be carried and the codec info
>>     they may use.
>
> Yes.
>
>> I'm just pointing out that for decomposed endpoints one 5-tuple may not
>> be enough. And the need for this might not be known a priori. One side
>> may be ready to add another stream for some purpose, and be willing/able
>> to use private signaling to negotiate that. But the other side may not
>> be able to handle that stream on the same 5-tuple. That in turn may
>> result in a need to negotiate SDP changes to establish an additional
>> 5-tuple that can then be used for that new stream.
>
> I think I see ... but not completely. Could you please post an SDP offer
> with BUNDLE and the corresponding answer?

The issue can be discussed without bundle, and without detailing the O/A.

Assume we have had an O/A already between X and Y, with a single (video) 
m-line, resulting in a single 5-tuple. Over this we have a single audio 
RTP stream and mediaStreamtrack.

The user of X interacts with the app, indicating a desire to receive 
another video stream from Y. This causes the app to trigger the JS in Y 
to set up another mediaStreamTrack. But the camera on Y has a separate 
IP address. So it is necessary to first set up another video m-line with 
the new address on Y's end. (X's end can use its single IP address.)

Once that is done the new mediaStreamTrack can be established on the new 
m-line.

How does this fit into No Plan?

> (This might be a good point to move this to mmusic)

Done.

	Thanks,
	Paul

> Thanks,
> Emil
>
>
>>>
>>>
>>>>
>>>>      Thanks,
>>>>      Paul
>>>>
>>>>> If you'd like for that signalling to be in SDP, I
>>>>> don't see any problem with it. However it would be best for this extra
>>>>> layer of SDP signalling to appear at either the application layer
>>>>> or in
>>>>> a signalling gateway (that is going to be there anyway).
>>>>>
>>>>> Does this make sense?
>>>>>
>>>>> Emil
>>>>>
>>>>>>
>>>>>> Further questions:
>>>>>>
>>>>>> I presume that you expect bandwidth in the SDP to be an aggregate
>>>>>> per-m-line, with application specific signaling for bandwidth at the
>>>>>> per-RTP-stream level?
>>>>>>
>>>>>>       Thanks,
>>>>>>       Paul
>>>>>>
>>>>>> On 5/29/13 2:59 PM, Emil Ivov wrote:
>>>>>>> Hey all,
>>>>>>>
>>>>>>> Based on many of the discussions that we've had here, as well as
>>>>>>> many
>>>>>>> others that we've had offlist, it seemed like a good idea to
>>>>>>> investigate
>>>>>>> a negotiation alternative that relies on SDP and Offer/Answer just a
>>>>>>> little bit less.
>>>>>>>
>>>>>>> The following "no plan" draft attempts to present one such approach:
>>>>>>>
>>>>>>> http://tools.ietf.org/html/draft-ivov-rtcweb-noplan
>>>>>>>
>>>>>>> The draft relies on conventional use of SDP O/A but leaves the
>>>>>>> intricacies of multi-source scenarios to application-specific
>>>>>>> signalling, with potentially a little help from RTP.
>>>>>>>
>>>>>>> Hopefully, proponents of Plans A and B would find that the
>>>>>>> interoperability requirements that concerned them can still be met
>>>>>>> with
>>>>>>> "no plan". Of course they would have to be addressed by
>>>>>>> application-specific signalling and/or signalling gateways.
>>>>>>>
>>>>>>> Comments are welcome!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Emil
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> rtcweb mailing list
>>>>>> [email-masked]
>>>>>> https://www.ietf.org/mailman/listinfo/rtcweb
>>>>>> .
>>>>>>
>>>>>
>>>>
>>>> .
>>>>
>>>
>>
>>
>

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 00:47:00 +0300

> In no-plan, What would the answer look like if you were receiving 2 video stream?

It would look like the above. Additional application-specific signalling 
would also say something like this:

{
      "firstStream":
      {
          "SSRC": "11111",
          "accept": "false"
      },
      "secondStream":
      {
          "SSRC": "22222",
          "accept": "true"
      },
      "thirdStream":
      {
          "SSRC": "333333",
          "accept": "true"
      },
}

Emil

-- 
https://jitsi.org

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 01:17:59 +0300

Hey Paul,

> the new address on Y's end. (X's end can use its single IP address.)

This is the part that I can't quite picture. How can Y's camera be on a 
separate IP, but still appear as a regular device to Y's javascript and 
still generate a regular stream track? All this with a generic browser?

Wouldn't it be much more likely that such a separate camera will be 
controlled by a proprietary interface that is not directly related to 
WebRTC?

If so, then the additional signalling for this camera would obviously 
need to be generated by Y's javascript (i.e. not the browser). That 
additional signalling can then be merged into the one from Y's browser 
by Y's JS or a gateway.

X would indeed need to handle an offer with two video m= lines at that 
point, but that's because they would negotiate different 5-tuples. So 
this is fine.

Another option would be to handle the new camera with an independent 
Offer/Answer that would yield a new PeerConnection. I don't know if that 
works for your case but I thought I'd mention it. In this case any 
additional information required for the synchronization of the two peer 
connections would be exchanged in an app-specific way.

Cheers,
Emil

Thanks,
Emil

-- 
https://jitsi.org

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 3 Jun 2013 17:47:25 -0700

On Mon, Jun 3, 2013 at 2:29 PM, Emil Ivov <[email-masked]> wrote:

>>              a) with Plan A and draft-roach-rtcweb-glareless-_**___add:
>>
>>

I mean demultiplexed by SSRC.

 If so, I've got some questions:
> however they wish. As far as browsers are concerned: the transmitter just
> sends whatever the application instructs. The receiver just handles
> whatever it can.

But there needs to be a complete set of API surfaces that allow
both sides to do the right thing.

>>
>
> What does "various" streams mean? If what you have in mind is: How does B
> tell A: "don't send me SSRCs 1, 2, 3 and 5" but keep 4 going, then B would
> send the list of SSRCs in app-specific signalling and A would stop sending
> them.

Uh, how does that work? Is there some API that I'm unaware of that
allows JS to get the SSRCs? And another one that allows the JS
to say "don't send SSRC A"?

It would be really helpful if you could provide a complete example of
exactly what each side does. I'm find if you just invent some
application-specific
signaling for this purpose, but what I'd like to see is exactly what API
calls each side makes.

-Ekr

---

From: Emil Ivov <[email-masked]>
Date: Tue, 04 Jun 2013 11:28:41 +0300

Hey Eric,

> On Mon, Jun 3, 2013 at 2:29 PM, Emil Ivov <[email-masked]> wrote:
<snip>
> I mean demultiplexed by SSRC.

Demuxing by SSRC, as in being able to distinguish one RTP stream from 
another is totally OK and there's no issue with this in No Plan. SSRCs 
will be different and packets will be assigned to different media 
streams. What isn't OK is the need to know specific SSRC values in order 
to perform the demuxing. This is not necessary for demuxing with No 
Plan. Do you agree?

>     This is something that one app needs to tell another app. They can
>     do so however they wish. As far as browsers are concerned: the
>     transmitter just sends whatever the application instructs. The
>     receiver just handles whatever it can.
>
>

Yes, hence section 7 in No Plan:

http://tools.ietf.org/html/draft-ivov-rtcweb-noplan-00#section-7

>     What does "various" streams mean? If what you have in mind is: How
>     does B tell A: "don't send me SSRCs 1, 2, 3 and 5" but keep 4 going,
>     then B would send the list of SSRCs in app-specific signalling and A
>     would stop sending them.
>
>
> allows JS to get the SSRCs?

I don't think there is. I believe you can get part of the information 
through the stats API but I don't think this is enough. Hence Section 7.

> And another one that allows the JS

As long as the JS knows the SSRC of a specific track it can just disable 
that specific track. There's no need to tell the browser "don't send 
SSRC A".

> signaling for this purpose, but what I'd like to see is exactly what API

Does the above answer your question, or would you like me to come up 
with an example of how these new APIs could look?

Emil

-- 
https://jitsi.org

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 4 Jun 2013 11:53:02 +0000

Hi,

>>> I don't understand why we would be talking about translating from one 
>>> of these plans to another. AFAIK they are *alternatives*, and that we 
>>> will at some point agree on one, and that will be the only one implemented.
>>
>> It has been pointed out that some existing non-WebRTC endpoints expect 
>> to get SDP "the Plan A way". Cullen asked how such devices could be 
>> made to communicate with "No Plan" browsers. This was my answer.
>>
>> Personally I haven't come across such devices so I can only speculate 
>> about the reasons why they work the way they work. Still I don't see 
>> why they shouldn't be able to talk to WebRTC browsers ... hence my answer.
>
> They clearly *don'* expect to get it the Plan A way (bundled).
> >
> I accept that there must be a variety of things that employ multiple
> (unbundled) m-lines, each on its own 5-tuple with one RTP stream.

I asked about that in a separate thread, but nobody replied, so I'll ask again :)

If the JS app receives multiple unbundled m- lines on the wire, does the JS app have to "convert" it into whatever Plan/Format supported by the API?

Regards,

Christer

>> On 6/3/13 1:28 PM, Emil Ivov wrote:
>>> Hey Cullen, Paul, all
>>>
>>> On 31.05.13, 22:19, Cullen Jennings (fluffy) wrote:
>>>> On 31.05.2013, at 12:23 PM, Emil Ivov <[email-masked]> wrote:
>>>>
>>>>> Certainly. Could you please post the SDP that you would like to 
>>>>> see translated in a way that's compatible with "No Plan"?
>>>>>
>>>>> Emil
>>>>
>>>> We can start with the SDP in plan A
>>>
>>> The example in "7.3. Many Videos" looks like a good start:
>>>
>>> http://tools.ietf.org/html/draft-roach-rtcweb-plan-a-00#section-7.3
>>>
>>> Here it is:
>>>
>>>      v=0
>>>      a=group:BUNDLE m0 m1 m2 m3
>>>
>>>      m=audio 56600 RTP/SAVPF 0 96
>>>      a=mid:m0
>>>      a=rtpmap:0 PCMU/8000
>>>      a=rtpmap:96 opus/48000
>>>      a=ptime:20
>>>      a=sendrecv
>>>      a=rtcp-mux
>>>      [ICE Candidates]
>>>
>>>      m=video 0 RTP/SAVPF 97 98
>>>      a=mid:m1
>>>      a=rtpmap:97 H264/90000
>>>      a=bundle-only
>>>      a=ssrc:11111 cname:45:5f:fe:cb:81:e9
>>>
>>>      m=video 0 RTP/SAVPF 97 98
>>>      a=mid:m2
>>>      a=rtpmap:97 H264/90000
>>>      a=bundle-only
>>>      a=ssrc:22222 cname:45:5f:fe:cb:81:e9
>>>
>>>      m=video 0 RTP/SAVPF 97 98
>>>      a=mid:m3
>>>      a=rtpmap:97 H264/90000
>>>      a=bundle-only
>>>      a=ssrc:333333 cname:45:5f:fe:cb:81:e9
>>>
>>>
>>> I. Talking to legacy
>>> In case you need to talk to an actual legacy (as in widely deployed 
>>> SIP) endpoint, the above would translate into a regular two-stream 
>>> call. Both Plan A and No Plan would lead to essentially the same 
>>> result (if we accept that older endpoints won't throw an exception 
>>> at the sight of 4 m= lines) so not much to discuss here.
>>>
>>> II. Talking to Plan A style endpoints
>>>
>>> If you need to talk to a Plan A endpoint you basically have the 
>>> following options:
>>>
>>> 1. You use JavaScript to prettify the "No Plan" SDP and turn it into 
>>> something that looks like "Plan A". Not my favourite option, but I 
>>> am sure some would like to use it. Maybe vendors of Plan A equipment 
>>> would even distribute JS libs that do this. It would basically all 
>>> come down to generating one ssrc attribute and two additional 
>>> m=lines and appending this to the existing SDP string.
>>>
>>> 2. The application retrieves SSRCs from the browser, adds additional 
>>> application-specific signalling to it and then sends the whole thing 
>>> to a signalling gateway. The gateway (which you would also have with 
>>> Plan
>>> A) would convert the SDP into what it needs it to be.
>>>
>>> The application specific signalling can look like this:
>>>
>>> {
>>>       "firstStream":
>>>       {
>>>           "SSRC": "11111",
>>>           "CNAME": "45:5f:fe:cb:81:e9"
>>>       },
>>>       "secondStream":
>>>       {
>>>           "SSRC": "22222",
>>>           "CNAME": "45:5f:fe:cb:81:e9"
>>>       },
>>>       "thirdStream":
>>>       {
>>>           "SSRC": "333333",
>>>           "CNAME": "45:5f:fe:cb:81:e9"
>>>       },
>>> }
>>>
>>> 3. In Plan B, section 3.1 talks about generating "Plan A" style SDP 
>>> with the help of .content properties. If browser vendors are willing 
>>> to implement support for this then I suppose it would be a third option.
>>>
>>> a) with Plan A and draft-roach-rtcweb-glareless-add:
>>> I am intentionally not going into how all plans would require 
>>> additional metadata that would place SSRC 1 left and 2 right as I 
>>> don't think this conveys any meaningful differences.
>>>
>>> Comments on the above are welcome. We could also move to another 
>>> scenario from the Plan A draft, if you believe that 7.3 is not 
>>> representative enough.
>>>
>>> Emil
>>>
>>>
>>
>>
>

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 4 Jun 2013 06:13:34 -0700

On Tue, Jun 4, 2013 at 1:28 AM, Emil Ivov <[email-masked]> wrote:

> another is totally OK and there's no issue with this in No Plan

Well, maybe. So, your claim is that multiple SSRCs on the same m-line are
to be rendered
separately? I note that that's *not* the view Cullen has  expressed.

> er one that allows the JS
> that specific track. There's no need to tell the browser "don't send SSRC
> A".

Well, given that the only mechanism you provide for addressing tracks
is SSRC, it'a hardly an advantage to not have a "don't send SSRC A"
API point, since that just means you need to go grovelling through the
MSTs for their SSRC values.

> Does the above answer your question, or would you like me to come up with
> an example of how these new APIs could look?
>

I would like to see a *complete* example with all the data flowing back and
forth
and the API calls (even if they are mocked up) that you think the sides will
make.

-Ekr

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 04 Jun 2013 12:07:30 -0400

On 6/3/13 6:17 PM, Emil Ivov wrote:

[snip]
> still generate a regular stream track? All this with a generic browser?

I'm not a browser guy, so maybe I am imagining things that are out of 
scope for webrtc, but maybe not.

At the very least you can have an app that gateways between a browser 
and some non-browser device. So assume that X is a browser and Y is not. 
What does the app tell the browser to do so that the new stream is on a 
different 5-tuple?

> controlled by a proprietary interface that is not directly related to

Can somebody who knows more about these things comment if this is 
entirely impossible for two devices accessible to the browser to be 
accessible only via distinct IP address/port, or if it might occur for a 
sufficiently sophisticated device?

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 04 Jun 2013 12:16:08 -0400

On 6/4/13 9:13 AM, Eric Rescorla wrote:

>     Demuxing by SSRC, as in being able to distinguish one RTP stream
>     from another is totally OK and there's no issue with this in No Plan
>
>
> Well, maybe. So, your claim is that multiple SSRCs on the same m-line
> are to be rendered

If we fold in *Colin's* comments, then demuxing by SSRC is separate from 
determining if/how/where to render that stream.

Is there a need for an API to associate multiple SSRCs with a single 
mediaStreamTrack? Or should it be up to the app to accept multiple 
mediaStreamTracks to render them to the same place?

	Thanks,
	Paul

>     As long as the JS knows the SSRC of a specific track it can just
>     disable that specific track. There's no need to tell the browser
>     "don't send SSRC A".
>
>

> I would like to see a *complete* example with all the data flowing back
> and forth
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic
>

---

From: Enrico Marocco <[email-masked]>
Date: Wed, 5 Jun 2013 11:11:19 +0200

> and the API calls (even if they are mocked up) that you think the sides will

Not trying to steal Emil's job, here's a rough example of how I (from a
web developer PoV) would like to be allowed to do it. Required API
extensions are in 5. and 8. (i.e. the .ssrc MST attributes).

Regular PC setup (this already well known):
http://goo.gl/cKfNf

Stream add:
http://goo.gl/ZNSdZ

Again, this is a *rough* example. The clear advantage (from my PoV of
course) is that, once the peer connection is established, the app does
not have to care about additional renegotiations, but in exceptional
cases (i.e. the new stream needs a new PT): no need to handle O/As
crossing on the wire, nor rollbacks.

The clear drawback is that, as it is now, it looks awfully ugly: tracks
are attached to the existing stream for transmission, but rendered on a
new stream on the receiving side. This is because the focus was on
minimizing the impact on the API. It could be probably improved with
more substantial changes, e.g. for allowing the instantiation of the
remote stream on the receiving side before the RTP packets actually
start flowing. Kind of what plan B does, but without the additional,
painful, O/A+O/A exchange.

Enrico

---

From: Emil Ivov <[email-masked]>
Date: Wed, 05 Jun 2013 12:48:45 +0200

On 05.06.13, 11:11, Enrico Marocco wrote:
> Not trying to steal Emil's job,

Steal away! The JS API part of this is something that we haven't even 
started designing yet so any feedback is welcome.

> here's a rough example of how I (from a

This is indeed along the lines of what I also had in mind. Stefan 
already made a comment on rtcweb about how obtaining SSRCs directly from 
a stream doesn't really fit right now as a stream may be attached to 
many, one or no peer connections. So I suppose the actual API would have 
to take this into account and get the SSRC from some object that 
represents a relation between a track and a specific peer connection. 
Some sort of a PeerTrackDescriptor ... or something like that.

Looking at the example itself I see it ends at bob asking Alice to stop 
sending a specific SSRC. I believe Eric's questions were also about that 
part, so here are a couple of more lines at the side of Alice:

onSigna(action, ssrc)
{
   if(action == 'stopTrack')
     stopTrack(ssrc);
}

stopStream(ssrc)
{
    localTracks[ssrc].stop();
}

In the above localTracks is a locally maintained array/map of the tracks 
Alice is currently sending. Obviously you could also just loop a "for" 
over the tracks in your streams but since you didn't want to "grovel" ...

Of course there's work to be done on the APIs but on the whole, Eric, 
does this answer your question?

Emil

-- 
https://jitsi.org

---

From: Emil Ivov <[email-masked]>
Date: Wed, 05 Jun 2013 18:30:01 +0200

Hey Paul,

> scope for webrtc, but maybe not.

Well I suppose a browser could always support a decoupled device if it 
were made available as local through some driver. Something like a 
virtual local device. In that case however, this driver would still have 
to bring the media to the local host so that it can be placed within a 
track, a stream and a peer connection. At this point it wouldn't matter 
to WebRTC that the device is actually decoupled, would it?

There is still the other option where this device is controlled by 
proprietary JS and for this you'd need to have your own signalling and 
your own control interfaces.

> and some non-browser device.

Let me make sure I understand this. This gateway app, is it running on 
X, is it on a box between X and Y, or is it a controlling entity of the 
decoupled Y?

> So assume that X is a browser and Y is not.

Without knowing the specifics of your scenario I'd say that you have the 
following options:

1) Y relays RTP from its decoupled camera toward X. In this case X is 
getting everything on the same 5-tuple, so no extra m= lines are 
necessary. (You'd have a similar situation in the case above where the 
camera is somehow presented as a virtual local device on Y)

2) Y just wants to handle signalling and it would like that video from 
the decoupled camera reaches X directly from there. Y also wants to have 
everything handled within the same Offer/Answer dialog. In this case:
2.1) When Y calls X, it would add a second video m= line in a second 
bundle (or with no bundle at all) when sending its offer to X
2.2) When X calls Y with a single lined offer, Y would also answer with 
a single m= line and would then re-Offer with two lines.

In both 2.1 and 2.2 the browser on X would see two independent, 
unbundled streams and handle them appropriately.

Is any of the above close to what you have in mind? Or is it completely off?

Emil

-- 
https://jitsi.org

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 06 Jun 2013 10:41:02 -0400

Hi Emil - more inline.

> X, is it on a box between X and Y, or is it a controlling entity of the

Consider it to be a web server coupled to a sip GW. So in the webrtc 
"triangle" view, X is a browser, Y is a SIP device, and the GW is the 
web server driving the broswer. For the moment lets assume that Y 
supports ICE, and bundle, and DTLS/SRTP, so that the GW need only be a 
signaling gw, not a media gw.

> camera is somehow presented as a virtual local device on Y)

(1) doesn't fit what I'm concerned with.

> bundle (or with no bundle at all) when sending its offer to X

This is more what I had in mind.
So at least you are ok with two separate video m-lines on different 
5-tuples in this case, where it is initiated from the sip device.
It wasn't evident to me if this fit into your model.

> a single m= line and would then re-Offer with two lines.

Lets consider a more complex variant of this scenario:

- the initial o/a is just for audio. That gets set up in obvious way.
   There is only an audio m-line. (Maybe in a degenerate bundle,
   maybe not.) I guess there is no data channel, because the gw doesn't
   expect sip devices to support one.

- then X (or the app in the gw driving X) decides to add video.
   So it creates second offer, including a video m-line bundled with
   the audio m-line.

- that offer goes to Y. But Y can't handle the audio and video as
   a bundle. In this case it can simply refuse the bundle while
   accepting the video unbundled.

If that is ok with No Plan then I think I am ok with it.

	Thanks,
	Paul

---

From: Emil Ivov <[email-masked]>
Date: Thu, 06 Jun 2013 17:30:36 +0200

Hey Paul,

> signaling gw, not a media gw.

OK.

> It wasn't evident to me if this fit into your model.

OK, I'll make sure it's clarified. The basic idea of No Plan is to use 
SDP for setting up transport channels and codec chains. The SDP way of 
setting up two transport channels is by having 2 m= lines so this has to 
be and *is* OK with No Plan.

>     expect sip devices to support one.

OK

>     So it creates second offer, including a video m-line bundled with

OK.

>     accepting the video unbundled.

Yes, this is OK.

Obviously, if the offer originated at the GW (and not X) then the 
session update has to reach X at some point because we are adding a new 
m= line.

> If that is ok with No Plan then I think I am ok with it.

Great!

Emil

-- 
https://jitsi.org