Subject: [MMUSIC] Scope of RTP payload types in BUNDLE?

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 13:29:41 +0100

There were a number of comments in the call last week, and on the list, about unique payload types in BUNDLE. I'd like to explore this further.

Case A: Within a single RTP session, I think we'd all agree that an offer that uses the same RTP payload type for two payload formats on a single m= line is problematic: 

   v=0
   o=alice 2890844526 2890844526 IN IP4 host.anywhere.com
   s=
   c=IN IP4 host.anywhere.com
   t=0 0
   m=audio 49170 RTP/AVP 96
   a=rtpmap:96 AMR-WB/16000
   a=rtpmap:96 G7291/16000

If this were done the receiver would have no way of distinguishing what payload format is meant by payload type 96. Accordingly, unique payload formats need to be used for each payload format.

Case B: If one were to use two separate m= lines on different ports, in the non-BUNDLE case, then the same RTP payload type can be reused without difficulty:

   v=0
   o=alice 2890844526 2890844526 IN IP4 host.anywhere.com
   s=
   c=IN IP4 host.anywhere.com
   t=0 0
   m=audio 49170 RTP/AVP 96
   a=rtpmap:96 AMR-WB/16000
   m=audio 49172 RTP/AVP 96
   a=rtpmap:96 G7291/16000

The implication is that there are two separate RTP sessions, which the receiver can distinguish based on the UDP port on which the packets are received. The mapping from RTP payload types to payload formats is done on a per-RTP session basis.

Case C: This is when we BUNDLE several m= lines on a single UDP port. On the call last week, there seemed to be agreement that the m= lines in such a BUNDLE group comprise a single RTP session (it also matches the definition of an RTP session from RFC 3550: the packets go to a single destination and the m= lines share a single SSRC space). To me this would suggest that the RTP payload type values MUST be unique across the m= lines (or, if the same RTP payload type is used in different m= lines, it MUST map to an identical payload format). The reason is that everything runs over one UDP port, and if a single RTP payload type is mapped to different payload formats, there's no way to distinguish which payload format is meant. This is essentially the same as case A above.

Given this, it's not clear to me that non-unique payload types make sense within a BUNDLE group. They work on different m= lines in the legacy case because those lines form different RTP sessions, and the RTP sessions can be used to scope the payload type to payload format mappings. I don't see that they work in BUNDLE, because there's a single RTP session, and so a single mapping from payload type to payload format.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 13:45:06 +0100

> If this were done the receiver would have no way of distinguishing what payload format is meant by payload type 96. Accordingly, unique payload formats need to be used for each payload format.

That should be "...unique payload types need to be used for each payload format" of course.

> _______________________________________________
> mmusic mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mmusic

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 27 May 2013 12:52:05 +0000

Hi,

> That should be "...unique payload types need to be used for each payload format" of course.

What if you have two m- lines, with identical encoding in both? Both represent the same payload format, don't they? :)

v=0
o=alice 2890844526 2890844526 IN IP4 host.anywhere.com
s=
c=IN IP4 host.anywhere.com
t=0 0
m=audio 49170 RTP/AVP 96
a=rtpmap:96 AMR-WB/16000
m=audio 49170 RTP/AVP ???
a=rtpmap:??? AMR-WB/16000

Regards,

Christer

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 13:59:04 +0100

> a=rtpmap:??? AMR-WB/16000

Not sure I get your point. You can have two different payload types that map to the same payload format in a single RTP session, since you can always distinguish what payload format is intended. You can't have the same payload type mapping to two different payload formats in a single RTP session, since you can't then infer what payload format was meant. 

-- 
Colin Perkins
http://csperkins.org/

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 27 May 2013 13:02:34 +0000

Hi,

>>>>  o=alice 2890844526 2890844526 IN IP4 host.anywhere.com  s=  c=IN 
>>>> IP4 host.anywhere.com
>>>>  t=0 0
>>>>  m=audio 49170 RTP/AVP 96
>>>>  a=rtpmap:96 AMR-WB/16000
>>>>  a=rtpmap:96 G7291/16000
>>>> 
>> What if you have two m- lines, with identical encoding in both? Both 
>> represent the same payload format, don't they? :)
>> 
>> v=0
>> o=alice 2890844526 2890844526 IN IP4 host.anywhere.com s= c=IN IP4 
>> host.anywhere.com
>> t=0 0
>> m=audio 49170 RTP/AVP 96
>> a=rtpmap:96 AMR-WB/16000
> Not sure I get your point. You can have two different payload types that map to the same payload format in a single RTP session, since
> you can always distinguish what payload format is intended. You can't have the same payload type mapping to two different payload 
> formats in a single RTP session, since you can't then infer what payload format was meant. 

Please not that both PTs map to the SAME payload format :)

Regards,

Christer

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 14:05:02 +0100

> Please not that both PTs map to the SAME payload format :)

I thought I addressed that in my reply.

-- 
Colin Perkins
http://csperkins.org/

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Mon, 27 May 2013 13:26:29 +0000

> If this were done the receiver would have no way of distinguishing what payload format is meant by payload type 96. Accordingly, unique payload formats need to be used for each payload format.

yes - agree of course

> The implication is that there are two separate RTP sessions, which the receiver can distinguish based on the UDP port on which the packets are received. The mapping from RTP payload types to payloa…

also agree of course 

> Case C: This is when we BUNDLE several m= lines on a single UDP port. On the call last week, there seemed to be agreement that the m= lines in such a BUNDLE group comprise a single RTP session (it…

Sure, suspect many people agree with you but the media pipeline is more than just the codec. So if I have 150 thumbnail videos that all using VP8, using one PT for all of them might be fine given they are all the same codec but they still might need SSRC to be able to figure out which window to render them in. I think that is all the algorithm I am proposing tries to be able to do.

---

From: Emil Ivov <[email-masked]>
Date: Mon, 27 May 2013 16:39:12 +0300

On May 27, 2013 4:27 PM, "Cullen Jennings (fluffy)" <[email-masked]>
wrote:

> Sure, suspect many people agree with you but the media pipeline is
>  more than just the codec. So if I have 150 thumbnail videos that all
> using VP8, using one PT for all of them might be fine given they are all
> the same codec but they still might need SSRC to be able to figure out
> which window to render them in.

How is tuis a demuxing issue?

Isn't this more closely related to application policies and hence subject
to application specific signalling? Of course that signalling can use SSRCs
but this does this make it an RTP issue?

Emil

--sent from my mobile

---

From: "Mo Zanaty (mzanaty)" <[email-masked]>
Date: Mon, 27 May 2013 13:47:06 +0000

The gist of Christer's question is: unique payload types just for unique payload formats, or also for unique sources? I think we all agree the former is true, and this seems to be Colin's point below. But Plan A proposes the latter and extends PT uniqueness to identify sources not just payloads.

Mo

On May 27, 2013, at 9:05 AM, "Colin Perkins" <[email-masked]> wrote:

-- 
Colin Perkins
http://csperkins.org/

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Mon, 27 May 2013 13:51:40 +0000

On May 27, 2013, at 7:39 AM, Emil Ivov <[email-masked]> wrote:

> 
> On May 27, 2013 4:27 PM, "Cullen Jennings (fluffy)" <[email-masked]> wrote:
> 
> Isn't this more closely related to application policies and hence subject to application specific signalling? Of course that signalling can use SSRCs but this does this make it an RTP issue?
> 
> Emil

I agree. I don't think it is an AVT* issues, I think it is a MMUSIC issues. One of the application signaling approaches often used for RTP is SDP. This is a SDP issue. In the plan A context, it can be roughly summarized as how do the RTP packets know which m-line in the SDP they are associated with. I think Ted put it better when he said which processing pipeline. Then things like like the content attribute in SDP can bind it to the applications intent. In the plan B context you might not use the word m-line but it is roughly the same problem.

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 14:59:51 +0100

On 27 May 2013, at 14:26, Cullen Jennings (fluffy) wrote:
> Sure, suspect many people agree with you but the media pipeline is more than just the codec. So if I have 150 thumbnail videos that all using VP8, using one PT for all of them might be fine given they are all the same codec but they still might need SSRC to be able to figure out which window to render them in.

Of course.

> I think that is all the algorithm I am proposing tries to be able to do. 

Then perhaps the disagreement is just about terminology, and what point in the reception process we're talking about demultiplexing.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 15:05:53 +0100

The SSRC identifies the source in RTP: that's fundamental to the design of the protocol. 

If Plan A wants to use unique PT values to determine how and where a source is rendered, that's fine, subject to the limited number of PT values. I don't think it's fine to redefine the PT as identifying the source.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 12:56:03 -0400

Colin,

As I suggested on another thread, ISTM that it should be possible to 
reuse the same PT to map to different payload formats in different 
m-lines of a bundle *if* there is some other characteristic declared in 
SDP and present in packets that can be used to associate the packet to 
one m-line. In that case, after picking the m-line, the mapping from PT 
to payload format for *that* m-line can be used. E.g.,

    v=0
    o=alice 2890844526 2890844526 IN IP4 host.anywhere.com
    s=
    c=IN IP4 host.anywhere.com
    t=0 0
    a=group:bundle X Y
    m=audio 49170 RTP/AVP 96
    a=mid:X
    a=ssrc:1111 cname:[email-masked]
    a=rtpmap:96 AMR-WB/16000
    m=audio 49172 RTP/AVP 96
    a=mid:Y
    a=ssrc:2222 cname:[email-masked]
    a=rtpmap:96 G7291/16000

	Thanks,
	Paul

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 27 May 2013 17:55:46 +0000

Hi,

> I thought I addressed that in my reply.

I am not sure you did - at least I didn't get it :)

Again, my understanding of what you said is that, for any payload format within an RTP session, the PT value has to be unique.

In the example above the same payload format, within the same RTP session, is used in two separate m- lines. But, I still can't use the same PT value for both m- lines, even if the payload format is the same, can I?

Regards,

Christer

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 19:10:48 +0100

> Again, my understanding of what you said is that, for any payload format within an RTP session, the PT value has to be unique.

Correct.

> In the example above the same payload format, within the same RTP session, is used in two separate m- lines. But, I still can't use the same PT value for both m- lines, even if the payload format i…

Why not? I don't see any problem mapping the same payload type to the exact same payload format in two different m= lines. Mapping the same payload type to two different payload formats is the problem.

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 19:29:14 +0100

Paul,

What would that other characteristic be? The only thing you have is the SSRC, and SDP doesn't scope "a=rtpmap:" lines to be per "a=ssrc:" line, and RTP certainly doesn't treat payload types as per SSRC.

Colin

On 27 May 2013, at 17:56, Paul Kyzivat wrote:
> Colin,
> 
> As I suggested on another thread, ISTM that it should be possible to reuse the same PT to map to different payload formats in different m-lines of a bundle *if* there is some other characteristic declared in SDP and present in packets that can be used to associate the packet to one m-line. In that case, after picking the m-line, the mapping from PT to payload format for *that* m-line can be used. E.g.,
> 
>   v=0

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 14:38:33 -0400

>> Again, my understanding of what you said is that, for any payload format within an RTP session, the PT value has to be unique.

Now you're both confused. What you are trying to say is that for any PT 
value withing an RTP session the payload format must be unique.

E.g., two different PTs may map to the same format, but one PT may not 
map to two different formats.

But I continue to argue that we could relax that, as long as there is 
something else that selects which PT mapping to use.

> Why not? I don't see any problem mapping the same payload type to the exact same payload format in two different m= lines. Mapping the same payload type to two different payload formats is the prob…

It depends. *If* you are trying to use the PT to associate packets to an 
m-line, then indeed you might need to use different PT values for the 
same payload format in order to accomplish that.

	Thanks,
	Paul

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 19:41:52 +0100

On 27 May 2013, at 19:38, Paul Kyzivat wrote:
> Now you're both confused. What you are trying to say is that for any PT value withing an RTP session the payload format must be unique.
> 
> E.g., two different PTs may map to the same format, but one PT may not map to two different formats.
> 
> But I continue to argue that we could relax that, as long as there is something else that selects which PT mapping to use.

Yes, but why?

> It depends. *If* you are trying to use the PT to associate packets to an m-line, then indeed you might need to use different PT values for the same payload format in order to accomplish that.

Sure, but no-one is arguing against that. The problem is using the same PT for different payload formats. 

-- 
Colin Perkins
http://csperkins.org/

---

From: Christer Holmberg <[email-masked]>
Date: Mon, 27 May 2013 18:43:58 +0000

Hi,

>Why not? I don't see any problem mapping the same payload type to the exact same payload format in two different m= lines.

But, when I receive media with that payload format, how do I know to which m- line it "belongs", as the same PT value is used for both m- lines?

Regards,

Christer

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 14:49:39 -0400

> What would that other characteristic be? The only thing you have is the SSRC, and SDP doesn't scope "a=rtpmap:" lines to be per "a=ssrc:" line, and RTP certainly doesn't treat payload types as per…

This is all part of defining the precise semantics of bundling!

In the example I gave below, what I am suggesting is that:

- SSRC 1111 is associated with m-line X, and SSRC 22222 is associated
   with m-line Y.

- When a packet is received, it must first be associated with
   an m-line. If it has SSRC=1111 then it can be associated with
   m-line X. Then, m-line X gives the mapping of PT 96 to audio
   and AMR. If it has SSRC=2222 then it can be associated with
   m-line Y. Then, m-line Y gives the mapping of PT 96 to audio
   and G.7291. (If a packet with some other SSRC is received
   then the mapping to an m-line is unknown, unless we introduce
   some other rule.)

- in some other example, if some PT is unique to a single m-line,
   then a packet with that PT can be associated to that m-line.

Have I made myself clear yet?

	Thanks,
	Paul

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 19:52:25 +0100

> But, when I receive media with that payload format, how do I know to which m- line it "belongs", as the same PT value is used for both m- lines?

You don't. If you care about that, you need to either use distinct PT values for each m= line, or signal SSRCs. 

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 14:55:52 -0400

> But, when I receive media with that payload format, how do I know to which m- line it "belongs", as the same PT value is used for both m- lines?

The PT can be viewed as method of last resort for associating a packet 
with an m-line, because the PT is the one thing that could be used for 
that association that you *must* specify in RTP. Everything else that 
*could* be used (e.g., SSRC) is optional.

So if nothing else is present in the SDP that could be used then the PTs 
need to be unique per-m-line. But if something else is present, then it 
is no longer necessary that the PTs be unique.

	Thanks,
	Paul

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 20:08:55 +0100

> Have I made myself clear yet?

Yes, but I think that's the wrong approach. 

When an RTP or RTCP packet arrives it first needs to be associated with an RTP source (step B in the diagram in http://www.ietf.org/mail-archive/web/mmusic/current/msg11251.html). That RTP source then, if you care about such things, can be associated with an m= line (step C in the diagram). 

You can use some combination of the payload type and a=ssrc: lines to perform the mapping at step C. That's fine. I'm arguing, however, that step B is important for RTP to work right, and that we can't just map to sources based on the payload type.

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 15:13:52 -0400

> You don't. If you care about that, you need to either use distinct PT values for each m= line, or signal SSRCs.

I assert that if you don't care about associating a packet to a 
particular m-line in a bundle that you don't need to use bundle.

So if you used bundle, you must care, and so there must be a way to 
achieve that association.

The reason you care is because doing so allows you to discover all the 
other things in that media section of the SDP that may be pertinent to 
this packet.

	Thanks,
	Paul

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 May 2013 20:24:25 +0100

On 27 May 2013, at 20:13, Paul Kyzivat wrote:
> I assert that if you don't care about associating a packet to a particular m-line in a bundle that you don't need to use bundle.

Sure.

> So if you used bundle, you must care, and so there must be a way to achieve that association.

Sure.

> The reason you care is because doing so allows you to discover all the other things in that media section of the SDP that may be pertinent to this packet.

Sure. 

But, again, let's keep the discussion of mapping RTP/RTCP packets onto RTP sources separate from the discussion of how to map the RTP sources onto m= lines. The former is well-defined by RTP, the latter is not yet defined. Proposals for solving the latter shouldn't start with "when a packet arrives, first do this", because that's the domain of the RTP source de-multiplexing process; rather they should consider "how do we map this RTP-level source (i.e., a sequence of RTP and RTCP packets identified by SSRC, whether signalled or not) onto an m= line". 

-- 
Colin Perkins
http://csperkins.org/

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 27 May 2013 15:27:57 -0400

Colin,

> Yes, but I think that's the wrong approach.

I think what I am talking about can be tweaked to fit with what you are 
suggesting. (This is where my limited understanding of RTP causes me 
trouble.) I've been viewing B & C as a single black box. I've no problem 
with decomposing it.

> When an RTP or RTCP packet arrives it first needs to be associated with an RTP source (step B in the diagram in http://www.ietf.org/mail-archive/web/mmusic/current/msg11251.html). That RTP source t…

OK. Based on the other things you have said, this seems fine.

> You can use some combination of the payload type and a=ssrc: lines to perform the mapping at step C. That's fine.

Then all the logic to bind the packet to an m-line can appear in step C.
I *think* it really only needs to be done once per SSRC, the first time 
the SSRC is seen. But maybe it can't be done by the first packet that 
arrives for that SSRC. (Certainly that may be so if the SSRC is first 
seen an an RTCP packet.)

> I'm arguing, however, that step B is important for RTP to work right, and that we can't just map to sources based on the payload type.

OK.

	Thanks,
	Paul