Subject: Re: [MMUSIC] Scope of RTP payload types in BUNDLE?

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:50:53 -0600

On May 27, 2013, at 8:05 AM, Colin Perkins <[email-masked]> wrote:

> The SSRC identifies the source in RTP: that's fundamental to the design of the protocol. 
> 
> If Plan A wants to use unique PT values to determine how and where a source is rendered, that's fine, subject to the limited number of PT values. I don't think it's fine to redefine the PT as identifying the source.
> 
> Colin

Again we might have confusions on the word source. In the sense Colin means source in which RTP speaking computer (or application on that computer), I agree with Colin. My defines in above sentence is pretty fluffy but the key thing is source seems pretty well defined in RTP and it's identified by the SSRC information (which might show up in a CSRC field). 

But I think some people use source sometimes to mean if the media originated from the microphone or the camera - that's not how RTP uses the word source but if people mean camera or microphone, then some of that might be learned from the PT.

> 
> 
> 
> On 27 May 2013, at 14:47, Mo Zanaty (mzanaty) wrote:
>> The gist of Christer's question is: unique payload types just for unique payload formats, or also for unique sources? I think we all agree the former is true, and this seems to be Colin's point below. But Plan A proposes the latter and extends PT uniqueness to identify sources not just payloads.
>> 
>> Mo
>> 
>> 
>> On May 27, 2013, at 9:05 AM, "Colin Perkins" <[email-masked]> wrote:
>> 
>> On 27 May 2013, at 14:02, Christer Holmberg wrote:
>>>>>>> There were a number of comments in the call last week, and on the list, about unique payload types in BUNDLE. I'd like to explore this further.
>>>>>>> 
>>>>>>> Case A: Within a single RTP session, I think we'd all agree that an offer that uses the same RTP payload type for two payload formats on a single m= line is problematic: 
>>>>>>> 
>>>>>>> v=0
>>>>>>> o=alice 2890844526 2890844526 IN IP4 host.anywhere.com  s=  c=IN 
>>>>>>> IP4 host.anywhere.com
>>>>>>> t=0 0
>>>>>>> m=audio 49170 RTP/AVP 96
>>>>>>> a=rtpmap:96 AMR-WB/16000
>>>>>>> a=rtpmap:96 G7291/16000
>>>>>>> 
>>>>>>> If this were done the receiver would have no way of distinguishing what payload format is meant by payload type 96. Accordingly, unique payload formats need to be used for each payload format.
>>>>>> 
>>>>>> That should be "...unique payload types need to be used for each payload format" of course.
>>>>> 
>>>>> What if you have two m- lines, with identical encoding in both? Both 
>>>>> represent the same payload format, don't they? :)
>>>>> 
>>>>> v=0
>>>>> o=alice 2890844526 2890844526 IN IP4 host.anywhere.com s= c=IN IP4 
>>>>> host.anywhere.com
>>>>> t=0 0
>>>>> m=audio 49170 RTP/AVP 96
>>>>> a=rtpmap:96 AMR-WB/16000
>>>>> m=audio 49170 RTP/AVP ???
>>>>> a=rtpmap:??? AMR-WB/16000
>>>> 
>>>> Not sure I get your point. You can have two different payload types that map to the same payload format in a single RTP session, since
>>>> you can always distinguish what payload format is intended. You can't have the same payload type mapping to two different payload 
>>>> formats in a single RTP session, since you can't then infer what payload format was meant. 
>>> 
>>> Please not that both PTs map to the SAME payload format :)
>> 
>> 
>> I thought I addressed that in my reply.
>> 
>> -- 
>> Colin Perkins
>> http://csperkins.org/
>> 
>> 
>> 
>> _______________________________________________
>> mmusic mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/mmusic
>> _______________________________________________
>> mmusic mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/mmusic
> 
> 
> 
> -- 
> Colin Perkins
> http://csperkins.org/
> 
> 
> 
> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:49:44 -0600

On May 27, 2013, at 1:13 PM, Paul Kyzivat <[email-masked]> wrote:

> On 5/27/13 2:52 PM, Colin Perkins wrote:
> 
>>>>> In the example above the same payload format, within the same RTP session, is used in two separate m- lines. But, I still can't use the same PT value for both m- lines, even if the payload format is the same, can I?
>>> 
>>>> Why not? I don't see any problem mapping the same payload type to the exact same payload format in two different m= lines.
>>> 
>>> But, when I receive media with that payload format, how do I know to which m- line it "belongs", as the same PT value is used for both m- lines?
>> 
>> You don't. If you care about that, you need to either use distinct PT values for each m= line, or signal SSRCs.
> 
> I assert that if you don't care about associating a packet to a particular m-line in a bundle that you don't need to use bundle.

I'm not sure this makes a huge difference to your argument but I don't think I agree with the above. 

It seems that even if you don't care which m-line the media get associated with, if you want to use a single port for all the m-line, you still need bundle. 

> 
> So if you used bundle, you must care, and so there must be a way to achieve that association.
> 
> The reason you care is because doing so allows you to discover all the other things in that media section of the SDP that may be pertinent to this packet.
> 
> 	Thanks,
> 	Paul

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:51:11 -0600

On May 27, 2013, at 1:24 PM, Colin Perkins <[email-masked]> wrote:

> 
> But, again, let's keep the discussion of mapping RTP/RTCP packets onto RTP sources separate from the discussion of how to map the RTP sources onto m= lines. The former is well-defined by RTP, the latter is not yet defined. Proposals for solving the latter shouldn't start with "when a packet arrives, first do this", because that's the domain of the RTP source de-multiplexing process; rather they should consider "how do we map this RTP-level source (i.e., a sequence of RTP and RTCP packets identified by SSRC, whether signalled or not) onto an m= line". 

I think this thread has convinced me that it was a mistake for my original proposal merged what was happening at what Colin had layer B and C in his digram. I'm roughly talking about how this gets connected up with the right m line at the application layer (layer C). 

I agree with the fact that RTCP and such is reported by SSRC as the layer B.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:51:28 -0600

On May 27, 2013, at 12:52 PM, Colin Perkins <[email-masked]> wrote:

> On 27 May 2013, at 19:43, Christer Holmberg wrote:
>>>>>>>> v=0
>>>> I am not sure you did - at least I didn't get it :)
>>>> 
>>>> Again, my understanding of what you said is that, for any payload format within an RTP session, the PT value has to be unique.
>>> 
>>> Correct.
>>> 

+1 

I think this pretty much nails it so if we could come up some text that said this, and did not have the terminology confusing I had introduced about what happens at layer B, I think that would be excellent.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:51:19 -0600

Imagine Alice sends  an offer containing 

On May 27, 2013, at 10:56 AM, Paul Kyzivat <[email-masked]> wrote:

> m=audio 49172 RTP/AVP 96
>  a=ssrc:2222 cname:[email-masked]
>  a=rtpmap:96 G7291/16000

Let say that an offer sent like the above. That means nothing about the SSRC for the packets that Alice will receive. And it is likely to receive RTP packets before it receives an answer in some important scenarios. 

So a RTP packet SSRC 3333 arrives. There's no information from the SSRC about what m-line it matches too (but RTCP for it can be sent). 

Now an answer from Bob arrives with 

m=audio 55555 RTP/AVP 96
 a=ssrc:4444 cname:[email-masked]
 a=rtpmap:96 G7291/16000

Now a packet with SSRC 4444 arrives. That might be from Bob, but here's the rub. It might be from Charlie. If there was an SSRC collisions between Bob and Charlie, it will be awhile before some SDP shows up that says that there was a collisions and now Bob is using SSRC 5555. 

This is all a disaster and the best solutions to it IMHO would be to define a new RTP Header extension that carries and identifier and that Alice can tell Bob in the SDP offer what value to put in the identifier for the RTP from Bob to Alice. 

I've see more than one system that have stuffed that identifier in the existing CSRC field which is sort of an awful hack and sort of very cool.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 3 Jun 2013 16:51:44 -0600

On May 27, 2013, at 1:27 PM, Paul Kyzivat <[email-masked]> wrote:

> 
> I think what I am talking about can be tweaked to fit with what you are suggesting. (This is where my limited understanding of RTP causes me trouble.) I've been viewing B & C as a single black box. I've no problem with decomposing it.

+1

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 4 Jun 2013 19:09:33 -0700

> So a RTP packet SSRC 3333 arrives. There's no information from the SSRC about what m-line it matches too (but RTCP for it can be sent). 
 
[BA] Yes, there is no SSRC info but you do have the PT.  Assuming that the RTP packet has a PT of 96 and that PT value is unique among all m lines (with BUNDLE), then you know what m line the RTP packet is referring to, and that it represents an audio packet. 

> Now a packet with SSRC 4444 arrives. That might be from Bob, but here's the rub. It might be from Charlie. If there was an SSRC collisions between Bob and Charlie, it will be awhile before some SDP…
 
[BA] I understand why this can happen if Bob is an RTP translator, but if Bob is another browser, why would packets from Charlie be received on a peerConnection between Alice and Bob? 

> This is all a disaster and the best solutions to it IMHO would be to define a new RTP Header extension that carries and identifier and that Alice can tell Bob in the SDP offer what value to put in…
 
[BA]  RTP extensions are optional, so even if you define one, you've got to be prepared to handle SSRCs that aren't pre-declared anyway, at least according to the RTP usage document.

---

From: Magnus Westerlund <[email-masked]>
Date: Wed, 5 Jun 2013 09:03:34 +0200

Hi,

Jumping into this late. But, I do Share almost all Colin Perkins has
written on this. And I think this discussion has gotten down to the gist
of things.

First of all, with Bundle you will have multiple m= blocks describing
the same RTP session, that combination must offer a consistent RTP
session definition. Thus, each used PT number must map to a unique
payload format and payload format configuration. However, I believe it
is important that we do allow the same PT == codec + payload format
configuration to be reused over multiple m= blocks, otherwise we will
shortly run into a scarcity issue with PT numbers.

> So a RTP packet SSRC 3333 arrives. There's no information from the SSRC about what m-line it matches too (but RTCP for it can be sent). 

Not only can RTCP be sent, the packets are identified to be a new stream
(due to new SSRC) and you know how to decode them. It is the binding
between the m= line and any application logic on how you intended to use
a stream that is missing.

This is a general problem and should be solved in a general way. Yes, we
could rely on requiring explicit SSRC signaling to bind it to a m=
block, or we can look at solutions that include the necessary
information somewhere where we are not having an issue with what arrives
first, i.e. we need something that can be included in the media plane,
i.e. RTP/RTCP and which can be bound to the signaling information
providing the application with the intended usage of a particular set of
streams.

> Now a packet with SSRC 4444 arrives. That might be from Bob, but
here's the rub. It might be from Charlie. If there was an SSRC
collisions between Bob and Charlie, it will be awhile before some SDP
shows up that says that there was a collisions and now Bob is using SSRC
5555.

Yes, there can be a SSRC collision. However, the RTP/RTCP layer will be
able to detect this, at the latest when two differnet CNAMEs claim to
use SSRC 4444.

> 
> This is all a disaster and the best solutions to it IMHO would be to
define a new RTP Header extension that carries and identifier and that
Alice can tell Bob in the SDP offer what value to put in the identifier
for the RTP from Bob to Alice.

I don't know if it is a disaster, as SSRC collision may occur but can be
sorted out and offers a glitch. I think a more likely glitch is due to
the general mismatch between relying on timely signaling of O/A and
dynamic behaviors among the media stream.

I think I am supporting the general idea of providing the "this is what
this stream is used for" in RTP/RTCP. My current thinking makes me
prefer a RTCP Source Description Item (SDES Item) which can also be
stuck in an RTP header extension to speed up acquiring the binding when
introducing a new stream and you can't really know that RTCP has made it
yet. That way each m= block, can have a short "usage tag" that is sent
as the SDES item for all RTP streams.

cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  +46 10 7148287
Färögatan 6                | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden| mailto: [email-masked]
----------------------------------------------------------------------

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Jun 2013 11:53:18 -0400

> It seems that even if you don't care which m-line the media get associated with, if you want to use a single port for all the m-line, you still need bundle.

I don't understand. Why do you want to use multiple m-lines on one port? 
If you don't need to associate the packets with a particular one of 
those m-lines, then why didn't you use a single m-line?

E.g., if your reason is because one is audio and another is video then 
you certainly want to associate the packets with one or the other.

ISTM that the whole point of Plan A is that each m-line is associated 
with a specific use/rendering, and so it is essential to associate 
incoming packets with a particular one of the m-lines.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Jun 2013 12:17:20 -0400

> Let say that an offer sent like the above. That means nothing about the SSRC for the packets that Alice will receive. And it is likely to receive RTP packets before it receives an answer in some im…

You keep saying that. But is it really true when ICE is used?
Need it be true even without ICE? After all, bundle is new usage, so we 
can demand new behavior when using it. So requiring a reliable 
provisional with an answer before sending media doesn't seem so 
unreasonable.

> So a RTP packet SSRC 3333 arrives. There's no information from the SSRC about what m-line it matches too (but RTCP for it can be sent).

My point is that if you choose to bundle, then there must be *some* 
mechanism to associate packets with m-lines. If you don't find the use 
of SSRC to be an acceptable solution, then feel free to construct your 
offer so that PTs are sufficient to do that association. But I don't see 
why the mechanism we specify must require that.

> Now a packet with SSRC 4444 arrives. That might be from Bob, but here's the rub. It might be from Charlie. If there was an SSRC collisions between Bob and Charlie, it will be awhile before some SDP…

Here you are assuming that the invite was forked, and that both forked 
endpoints send early media, and they manage to have an SSRC collision on 
the early media? (And both Bob and Charlie had the winning ticket on 
last night's Power Ball???)

We are supposed to worry about that case???

> This is all a disaster and the best solutions to it IMHO would be to define a new RTP Header extension that carries and identifier and that Alice can tell Bob in the SDP offer what value to put in…

That is another solution.
But do we want to require that all media providers do this?
Wouldn't it be less onerous to simply require an early answer?

I'm concerned with requiring the sender to insert something in the media 
that is specified by the receiver. That makes it harder to distribute 
the same media to multiple destinations.

I do think we want a header extension that contains something 
independent of SSRC that may be used to associate related packets. (That 
can be used by clue to carry capture-id.) But I don't think it needs to 
be defined by the receiver.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 5 Jun 2013 16:23:57 +0000

Hi,

I think there are many reasons why one would want to use different m- lines. Eventhough the transport characteristics are the same (if you use BUNDLE, that is), there other things that may be different. Just to give a simple example: the direction attribute value, content attribute value, and anything related to the "scope"/"context" of the m- line.

Regards,

Christer

-----Alkuperäinen viesti-----
Lähettäjä: [email-masked] [mailto:[email-masked]] Puolesta Paul Kyzivat
Lähetetty: 5. kesäkuuta 2013 18:53
Vastaanottaja: Cullen Jennings
Kopio: [email-masked]
Aihe: Re: [MMUSIC] Scope of RTP payload types in BUNDLE?

I don't understand. Why do you want to use multiple m-lines on one port? 
If you don't need to associate the packets with a particular one of those m-lines, then why didn't you use a single m-line?

E.g., if your reason is because one is audio and another is video then you certainly want to associate the packets with one or the other.

ISTM that the whole point of Plan A is that each m-line is associated with a specific use/rendering, and so it is essential to associate incoming packets with a particular one of the m-lines.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Jun 2013 12:47:53 -0400

> I think there are many reasons why one would want to use different m- lines. Eventhough the transport characteristics are the same (if you use BUNDLE, that is), there other things that may be diffe…

Christer - I suspect we are on the same page.

I agree there are many reasons one might want to bundle m-lines.
But I assert that every one of those reasons will also require that you 
be able to associate incoming packets with a particular one of those 
m-lines.

I don't understand why this is in question. I originally stated it as a 
truism because I thought it was self evident.

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Wed, 5 Jun 2013 17:01:25 +0000

Hi,

>> I think there are many reasons why one would want to use different m- lines. Eventhough the transport characteristics are the 
>> same (if you use BUNDLE, that is), there other things that may be different. Just to give a simple example: the direction attribute value, content attribute value, and anything related to the "scope"/"context" of the m- line.
>
> But I assert that every one of those reasons will also require that you be able to associate incoming packets with a particular one of those m-lines.
>
> I don't understand why this is in question. I originally stated it as a truism because I thought it was self evident.

Me too. My assumption has from the start been that we need *A* mechanism to associate media with m- lines, as there might be m- line specific characteristics (I haven't gone through all SDP attributes etc with that in mind, though, so I can't give concrete examples at this point) that are needed in order to properly process the media. I thought that relying on PT and/or SSRC/CNAME would cover all cases, but obviously it doesn't. 

For example, I PERSONALLY think one can assume to receive an SDP Answer (including SSRC/CNAME information etc) before media is received. But, as Cullen has another opinion I don't want to waste time on arguing about that - and I am fully aware what 3264 says about receiving media before the SDP answer :)

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Jun 2013 18:18:12 -0400

> For example, I PERSONALLY think one can assume to receive an SDP Answer (including SSRC/CNAME information etc) before media is received. But, as Cullen has another opinion I don't want to waste tim…

My understanding of this has been evolving as the discussion has 
continued. That is why I was eventually motivated to start the thread
"Associating packets with m-lines in a bundle" because I thought it was 
a way to crystalize that issue without having to initially agree on 
*how*. But it turns out that apparently it still isn't universally 
agreed that its necessary to do that association!

The "Plan" discussions make clear that how you intend to use bundling 
impacts how hard the association will be and what mechanisms for doing 
it might be sufficient. For instance, if you never bundle more than one 
m-line of each top level media type (one audio, one video, ...) then PT 
will probably be sufficient to get the job done. If you use Plan A with 
a separate m-line for each of 64 video thumbnails, then maybe not. If 
you use Plan A with CLUE, dynamic switched captures will need something 
else.

So I'm now thinking that BUNDLE with have to be defined without 
agreement on how the association is done. Rather, it will be up to uses 
of bundle to specify that.

	Thanks,
	Paul

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 6 Jun 2013 12:41:28 +0700

inline but I agree

> [BA] Yes, there is no SSRC info but you do have the PT.  Assuming that the RTP packet has a PT of 96 and that PT value is unique among all m lines (with BUNDLE), then you know what m line the RTP p…

yes - so for the layer C or sorting out the which m-line it goes to, I like the PT if it's unique and if you need more than that then PT + SSRC info you know. 

> [BA] I understand why this can happen if Bob is an RTP translator, but if Bob is another browser, why would packets from Charlie be received on a peerConnection between Alice and Bob? 

agree - I should have said it but I was thanking about the hangout type situation with perhaps 1000s of clients

> [BA]  RTP extensions are optional, so even if you define one, you've got to be prepared to handle SSRCs that aren't pre-declared anyway, at least according to the RTP usage document. 

agree

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 6 Jun 2013 12:41:41 +0700

On Jun 5, 2013, at 11:17 PM, Paul Kyzivat <[email-masked]> wrote:

>>  And it is likely to receive RTP packets before it receives an answer in some important scenarios.
> 
> Need it be true even without ICE? After all, bundle is new usage, so we can demand new behavior when using it. So requiring a reliable provisional with an answer before sending media doesn't seem so unreasonable.

I'm assuming that some applications will bring up a data channel (using ICE), as soon as the far end starts alerting the user so this will be done by the time the far end users agrees to send media. At that point it would be nice to start sending media with no clipping. That allows all the ICE setup time to disappear from the media clipping latency problem. 

I realize lots of applications will not work this way.

---

From: Cullen Jennings <[email-masked]>
Date: Thu, 6 Jun 2013 12:41:53 +0700

> For example, I PERSONALLY think one can assume to receive an SDP Answer (including SSRC/CNAME information etc) before media is received. But, as Cullen has another opinion I don't want to waste tim…

In fairness here we have some different type of deployments environments. Christer is often concerned about an environment where using SIP, things like PRACK are used which allows one to get an answer before media. IN fact media is probably gated and won't work if there is not an answer before the media. I am often concerned about deployments environments where PRACK is not supported but media is not gated.

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 6 Jun 2013 04:04:32 -0700

>> ISTM that the whole point of Plan A is that each m-line is associated with a specific use/rendering, and so it is essential to associate incoming packets with a particular one of the m-lines.

[BA] The above implies some interesting interactions with BUNDLE. For example, in H.264/SVC, in the unicast case, multiple layers are transported in the same RTP session, with or without BUNDLE. Therefore if each layer has its own m line, both sides MUST support BUNDLE to enable negotiation of layered coding even if there is no interest in bundling multiple media on the same RTP port. This is not true of Plan B ( or No Plan).

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 07 Jun 2013 12:39:45 -0400

> I realize lots of applications will not work this way.

We're mixing apples and oranges here I think.

In a pure sip environment there probably won't be a data channel, except 
for CLUE. But perhaps CLUE is what we should be considering when we are 
talking about bundle in a pure sip environment.

Even webrtc won't always be using a data channel.

In cases where there is a data channel, I don't know how much data will 
be exchanged over it before the "alerting" is done and the connection 
acknowledged. I certainly want that to be very constrained. So I'm not 
sure what the relevance of the data channel is to this discussion.

Reliable provisionals only apply when SIP is involved, on at least half 
of the call. But it doesn't matter, because for RTCWEB the equivalent 
thing exists - simply sending an answer early on. Once that has 
happened, the UAC can know what SSRCs to expect from the other end and 
use that as part of the process of associating packets with m-lines.

So my question is still: is it unreasonable to expect an answer to be 
returned and confirmed before the UAS starts sending media? This can 
happen as early as the commencement of alerting at the UAS. That means 
requiring 1.5 round trips over the signaling channel before starting to 
send. It will very likely require more round trips than that over the 
media channel (because of ICE) before media can flow in any case.

Also, until the UAC receives an answer it can't do ICE. So when ICE is 
in use aren't you guaranteed to have had a chance to get those SSRCs 
from the UAS?

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 07 Jun 2013 12:42:12 -0400

> In fairness here we have some different type of deployments environments. Christer is often concerned about an environment where using SIP, things like PRACK are used which allows one to get an ans…

There are still environments where PRACK is not *supported*???
(As opposed to being supported but not being used.)

If so, I'll bet those environments also don't support BUNDLE. :-)
And then can't we demand that to support BUNDLE you must also support PRACK?

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Fri, 7 Jun 2013 15:51:34 -0700

> in use aren't you guaranteed to have had a chance to get those SSRCs 
 
[BA] I think we need to distinguish the browser-browser use case from browser-gateway.  By "can't do ICE" you mean "can't start connectivity checks".  However, connectivity checks only restrict the ability to send, not receive.  In the browser-browser use case, this should restrict either side from sending until the Answer is received. 
 
However, in the browser-gateway case the gateway may only support ICE lite, so it will not send connectivity checks, only respond to them.  As a result, the gateway can send media once it receives the Offer; it is not restricted from doing so by having to send and receive a response to a connectivity check.  Of course, it will not have media to send until it actually gets some, but in the event that it receives early media, it does seem possible that it could forward that media on to the browser before the Answer were to arrive. 
 
I guess the question is how much we care about preventing clipping in a case like this.  Note that Section 4.3.2 of the use case document does hint at this, although it (and the requirements) don't explicitly point to a need to support of "early media". 
 4.3.2.1.  Description

   Alice uses her web browser with a service something like Skype to be
   able to phone PSTN numbers.  Alice calls 1-800-gofedex.  Alice should
   be able to hear the initial prompts from the fedex IVR and when the
   IVR says press 1, there should be a way for Alice to navigate the
   IVR.

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 07 Jun 2013 19:15:34 -0400

On 6/7/13 6:51 PM, Bernard Aboba wrote:
> [BA] I think we need to distinguish the browser-browser use case from
> browser-gateway.  By "can't do ICE" you mean "can't start connectivity
> checks".  However, connectivity checks only restrict the ability to
> send, not receive.  In the browser-browser use case, this should
> restrict either side from sending until the Answer is received.
>
> However, in the browser-gateway case the gateway may only support ICE
> lite, so it will not send connectivity checks, only respond to them.  As
> a result, the gateway can send media once it receives the Offer; it is
> not restricted from doing so by having to send and receive a response to
> a connectivity check.  Of course, it will not have media to send until
> it actually gets some, but in the event that it receives early media, it
> does seem possible that it could forward that media on to the browser
> before the Answer were to arrive.
>
> I guess the question is how much we care about preventing clipping in a
> case like this.  Note that Section 4.3.2 of the use case document does
> hint at this, although it (and the requirements) don't explicitly point
> to a need to support of "early media".

There is a big difference between clipping for one second until the 183 
is received, and clipping until the phone is answered and a 200 received.

But I was suggesting that we couple a requirement for PRACK with BUNDLE, 
and say that media should not be sent until the PRACK is received. So 
then there won't be clipping at all.

(According to a formal definition of clipping. In practice it doesn't 
matter whether the audio was dropped by the receiver or dropped by the 
sender.)

	Thanks,
	Paul

> 4.3.2.1
> <http://tools.ietf.org/html/draft-ietf-rtcweb-use-cases-and-requirements-10#section-4.3.2.1>.
> Description
>
> able to phone PSTN numbers. Alice calls 1-800-gofedex. Alice should be
> able to hear the initial prompts from the fedex IVR and when the IVR
> says press 1, there should be a way for Alice to navigate the IVR.
>
>
>

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 8 Jun 2013 11:11:28 +0700

On Jun 7, 2013, at 11:39 PM, Paul Kyzivat <[email-masked]> wrote:

> In a pure sip environment there probably won't be a data channel, except for CLUE. But perhaps CLUE is what we should be considering when we are talking about bundle in a pure sip environment.
> 
> In cases where there is a data channel, I don't know how much data will be exchanged over it before the "alerting" is done and the connection acknowledged. I certainly want that to be very constrained. So I'm not sure what the relevance of the data channel is to this discussion.

I don't think it is constrained. A JS app can bring up the data channel and never alert or get any consent from the end user if there is no camera or microphone involved. People are thinking of using this to do P2PLive  style video distribution among other things. If you go back to my high /  mid / low path slides from a way back, you can see I want to bring up a data channel than use that data channel to pass back and forth any updated offer / answers so that they are direct and don't add extra load to the web server. All of this could happen before any alerting even started.

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 8 Jun 2013 11:11:33 +0700

> And then can't we demand that to support BUNDLE you must also support PRACK?

I think you will find the people that have not implemented PRACK still don't want to implement PRACK. I know there is no way I would want to mandate the complexity of PRACK just to do bundle.

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 09 Jun 2013 13:03:32 -0400

> I don't think it is constrained. A JS app can bring up the data channel and never alert or get any consent from the end user if there is no camera or microphone involved. People are thinking of usi…

I agree its not technically constrained.

But still I think the app needs to be circumspect in the sort of data it 
passes prior to getting some sort of confirmation from the user.

Stuff that is just used to finish setup of communication is probably 
innocuous. But even that ought to be subject to policy control.
I may wish to appear unavailable until I answer the alert.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 09 Jun 2013 13:07:23 -0400

> I think you will find the people that have not implemented PRACK still don't want to implement PRACK. I know there is no way I would want to mandate the complexity of PRACK just to do bundle.

They are willing to implement BUNDLE, and probably ICE and DTLS/SRTP, 
and perhaps RTCWEB, but they find PRACK undesirable to implement???

I have no sympathy!
Its time to join the 21st century.

I see no reason to struggle for ways to work around that if mandating 
PRACK would solve the problem.

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 9 Jun 2013 10:32:30 -0700

> PRACK would solve the problem.

[BA]  "solving the problem" requires more than just PRACK.  In a WebRTC implementation, I believe that the requirement is to be able to receive media before setRemoteDescription is called.   Unless that  functionality (which is not specifically called out in the use case document) is implemented, it doesn't matter whether PRACK is supported or not.

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 9 Jun 2013 10:48:53 -0700

> is received, and clipping until the phone is answered and a 200 received.

[BA] So you're saying it is sufficient to enable media once a 183 is received, but not before?  I believe that is consistent with the current WebRTC API.   The use case document isn't clear that this is the bar that is being set, though.  

One issue is what the WebRTC gateway will do if it gets media before the 183.    If it just forwards it on, then it will be dropped by the browser until the 183 shows up and setRemoteDescription is called.  Should it buffer it until the 183 arrives?

> then there won't be clipping at all.

[BA] Personally, I don't see PRACK as being required for BUNDLE, at least in WebRTC uses. However, a WebRTC/SIP gateway probably should support PRACK.

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 09 Jun 2013 17:48:04 -0400

On 6/9/13 1:32 PM, Bernard Aboba wrote:
> [BA]  "solving the problem" requires more than just PRACK.  In a WebRTC
> implementation, I believe that the requirement is to be able to receive
> media before setRemoteDescription is called.   Unless that
> functionality (which is not specifically called out in the use case
> document) is implemented, it doesn't matter whether PRACK is supported
> or not.

I don't think it means that at all.

The PRACK means a reliable provisional has been sent, and so SDP carried 
is a true answer rather than a hint about a future answer.

If gatewaying between SIP and RTCWEB, that means that an ANSWER can be 
delivered rather than a PRANSWER. So AFAIK that means 
setRemoteDescription can be called.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 09 Jun 2013 18:02:15 -0400

On 6/9/13 1:48 PM, Bernard Aboba wrote:
> [BA] So you're saying it is sufficient to enable media once a 183 is
> received, but not before?  I believe that is consistent with the current
> WebRTC API.   The use case document isn't clear that this is the bar
> that is being set, though.

I'm suggesting that in the case of BUNDLE, there be a requirement that 
media not be sent until an answer is known to have been received by the 
other end. Sending a reliable 183 and awaiting the PRACK before sending 
media would satisfy that. Presumably something similar can serve in a 
pure WebRTC environment.

That will ensure no clipping.

The objection I hear to this is that some don't implement PRACK. I'm 
just saying that we could require support for PRACK with support for 
BUNDLE. Then that argument becomes bogus.

Then we can say that it is ok if something in the answer is needed for 
the offerer to demux incoming media.

> One issue is what the WebRTC gateway will do if it gets media before the
> 183.    If it just forwards it on, then it will be dropped by the
> browser until the 183 shows up and setRemoteDescription is called.
> Should it buffer it until the 183 arrives?

My proposal is that it should not transmit media until it knows the 
answer has been received. I don't care how it achieves that, but I 
imagine dropping it would be a good choice.

Or, if people prefer, we could allow it to be sent, but allow the other 
end to drop it. But still expect the early answer so that the period 
when media is lost is brief.

> [BA] Personally, I don't see PRACK as being required for BUNDLE, at
> least in WebRTC uses. However, a WebRTC/SIP gateway probably should
> support PRACK.

PRACK wouldn't be required in JSEP. But the equivalent would be expected 
- a quick ANSWER, before media is sent.

	Thanks,
	Paul

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 10 Jun 2013 12:48:16 +0900

+1 

and "work with 1-800-gofedex" is a deliberate wording so we can make sure we work with at least one of the key use cases but at the same time not drag in every weird corner case that has been imagined for early media.

---

From: Cullen Jennings <[email-masked]>
Date: Mon, 10 Jun 2013 12:48:19 +0900

On Jun 10, 2013, at 2:07 AM, Paul Kyzivat <[email-masked]> wrote:

> They are willing to implement BUNDLE, and probably ICE and DTLS/SRTP, and perhaps RTCWEB, but they find PRACK undesirable to implement???
> 
> I have no sympathy!

The other point of of vies is that PRACK is very complicated, has had many interoperable issues, and for many deployments, unnecessary, we should join the 21 century and deprecate it. 

Now you are unlikely to chance your opinion, I am unlikely to change mine, and there are many people in both camps. I will note a huge percentage of SIP equipment does not support it or has an implantation that does not meet the requirements of the RFC - particularly in the many unspecified corner cases. I will also note that this discussion if not new. There is a reason PRACK is an extension to SIP and not part of 3261. It's optional - and lots of people don't need or want the option. 

My proposed way to move forward is that we admit there are large use bases in both camps and make sure we can do something that works for both theses camps. So far we have managed to do that just fine.

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Mon, 10 Jun 2013 10:47:05 +0000

"has an implantation that does not meet the requirements of the RFC - particularly in the many unspecified corner cases"

Does not make sense!

Keith

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Cullen Jennings
> Sent: 10 June 2013 04:48
> To: Paul Kyzivat
> Cc: [email-masked] WG
> Subject: Re: [MMUSIC] Scope of RTP payload types in BUNDLE?
> 
> 
> >> I think you will find the people that have not implemented PRACK still
> don't want to implement PRACK. I know there is no way I would want to
> mandate the complexity of PRACK just to do bundle.
> >
> The other point of of vies is that PRACK is very complicated, has had many
> interoperable issues, and for many deployments, unnecessary, we should
> join the 21 century and deprecate it.
> 
> Now you are unlikely to chance your opinion, I am unlikely to change mine,
> and there are many people in both camps. I will note a huge percentage of
> SIP equipment does not support it or has an implantation that does not
> meet the requirements of the RFC - particularly in the many unspecified
> corner cases. I will also note that this discussion if not new. There is a
> reason PRACK is an extension to SIP and not part of 3261. It's optional -
> and lots of people don't need or want the option.
> 
> My proposed way to move forward is that we admit there are large use bases
> in both camps and make sure we can do something that works for both theses
> camps. So far we have managed to do that just fine.
>

---

From: "Stach, Thomas" <[email-masked]>
Date: Mon, 10 Jun 2013 12:40:41 +0000

> -----Ursprüngliche Nachricht-----
> Von: [email-masked] [mailto:[email-masked]] 
> Im Auftrag von Paul Kyzivat
> Gesendet: Montag, 10. Juni 2013 00:02
> An: Bernard Aboba
> Cc: [email-masked]
> Betreff: Re: [MMUSIC] Scope of RTP payload types in BUNDLE?
> 
Just as well, we could require ICE together with BUNDLE and 
re-use how ICE deals with reliability of provisional reponses.
In the RTCWEB this would be a non-issue. 
Environments that don't support ICE could still require PRACK.

Regards
Thomas

---

From: "Stach, Thomas" <[email-masked]>
Date: Mon, 10 Jun 2013 12:20:49 +0000

> Gesendet: Sonntag, 09. Juni 2013 19:07
> An: Cullen Jennings
> Cc: [email-masked]; Christer Holmberg
> > I think you will find the people that have not implemented 
> PRACK still don't want to implement PRACK. I know there is no 
> way I would want to mandate the complexity of PRACK just to do bundle.
> 

When ICE development there were similar arguments with respect to PRACK.
The consensus was the PRACK is recommended to be used.
If PRACK was not supported the answer is repeated until the first connectivity checks arrived at the answerer, meaning that the answer was received at the offerer.

Regards 
Thomas

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Jun 2013 12:42:54 -0400

> and "work with 1-800-gofedex" is a deliberate wording so we can make sure we work with at least one of the key use cases but at the same time not drag in every weird corner case that has been imagi…

Here I guess you mean WebRTC work with 1-800-gofedex, right?

That will either require the fedex web site to directly support WebRTC, 
or else somebody to deploy a WebRTC-pstn GW.

I would expect that fedex would be one of the first to want to support 
WebRTC. And then they can make it work directly.

I don't know who will deploy a general WebRTC-pstn GW, but probably some 
will. Then there are sub-cases for that:

- would the GW be able to directly access 1-800-gofedex via a sip
   trunk that supports DTLS/SRTP and ICE, so that it need not process
   the media itself, and is dependent on it being processed downstream?

- or will the WebRTC-pstn GW be processing media, so it can do the
   ICE and DTLS/SRTP and then transition to a "normal" SIP trunk or
   pstn GW?

The main thing is to get a quick answer so that early media processing 
is not delayed. I don't see why 1-800-fedex presents any special problems.

	Thanks,
	Paul

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 10 Jun 2013 12:45:00 -0400

> Environments that don't support ICE could still require PRACK.

The key is to ensure that the offerer gets an answer in a timely way to 
avoid clipping. PRACK is one way. ICE is another. We could require 
either/or.

	Thanks,
	Paul

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 11 Jun 2013 15:00:35 +0900

Sorry - trying to say 

Several of the implementations of 100rel have bugs in the case of where there is an outstanding transaction and they get an event that might cause them to want to start a second transaction before the first transaction completes.

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 11 Jun 2013 10:46:12 -0400

> Several of the implementations of 100rel have bugs in the case of where there is an outstanding transaction and they get an event that might cause them to want to start a second transaction before…

We are in real trouble if we say a feature can't be used because some 
implementations of it are buggy. We might as well all go home.

(It's a different story if nobody is able to construct a working 
implementation.)

	Thanks,
	Paul