Subject: RTCP Retransmission

From: Koichi Yano <[email-masked]>
Date: Thu, 16 Dec 1999 09:26:02 -0800

We are working on refining the RTCP retransmission draft.
I have questions about how to negotiate the type of
retransmission--whether or not retransmissions will be
used, and if so what kind of retransmission
algorithms/behavior will you use.

In the IETF meeting, it was suggested that a new payload type should be
defined in case of deployment of RTCP retransmission.
Is there another possible mechanism for negotiation about
RTCP deployment?

Is it necessary to define a new payload type for each
existing type?  In other words, do we need to
define new dynamic payload types like H261-rx,
H263-rx, DAT12-rx, etc?  When we want to add
retransmission on exisiting static payload type, could
it (a new payload type) become a problem for being
deployed on original implementation?

Koichi Yano

---

From: Nikki Cranley <[email-masked]>
Date: Fri, 17 Dec 1999 09:22:28 +0000

Hi,

I am looking at RTCP-Rx also with respect to MPEG-4. In MPEG-4 the BIFS and
IOD are very important for establishing the transmission and replay of the
MPEG-4 file. I am thinking of using ways to implement the RTCP-Rx to ensure
that these files are received.
I think that there should be new payload types defined to reflect different
media types etc. The intended purpose of RTCP-Rx is selective
retransmisison  and so there need to be selection criteria otherwise its
like TCP.

Regards,
Nikki

---

From: Colin Perkins <[email-masked]>
Date: Fri, 17 Dec 1999 16:26:02 +0000

>Is there another possible mechanism for negotiation about

You could define a new RTP profile, which uses the existing set of payload
formats but adds the retransmission scheme in RTCP (and, perhaps, also adds
congestion control).

That would be signalled in SDP, much like:
	m=audio 49000 RTP/rx-profile 0
with the "0" representing the payload type (PCMU audio, in this case), and
the "RTP/rx-profile" indicating the new profile. 

Colin

---

From: Colin Perkins <[email-masked]>
Date: Fri, 17 Dec 1999 16:30:22 +0000

>retransmisison  and so there need to be selection criteria otherwise its

For unicast, I wonder if TCP is not the correct means of transporting this
information, and would be interested in arguments why not.  For multicast, 
I remind you of RFC2357 and the work in RMT, which might be appropriate. 

Cheers,
Colin

---

From: Nikki Cranley <[email-masked]>
Date: Fri, 17 Dec 1999 16:41:53 +0000

Hi Colin -

The problem with TCP is that for real-time or near real time transmission its
not feasible to use TCP as there may be lost packets/delays etc.and
retransmission of these will hinder the real-time ability - so RTP/UDP/IP is
used.
But in the case of MPEG-4 - there are some important packets that probably
shouldn't be lost - imagine if the packet with all the decoding information got
lost - well that would render everything else useless. I am looking at this only
from the perspective of unicast anyway - plus if we were to extend this to VR -
then RTCP-Rx could be used to dictate user interaction as there is no scope for
this in RTP/RTCP as far as I know. It would only be needed from client to
server.
I'm only new to this but thats what came to mind. I was thinking along the lines
of say incorporating media specific functionality into the RTCP-Rx.

Rgds,
Nikki

---

From: Colin Perkins <[email-masked]>
Date: Fri, 17 Dec 1999 16:52:05 +0000

Hi,

>of say incorporating media specific functionality into the RTCP-Rx.

I was envisaging that RTCP-Rx would be used for streams which need to be
resilient to loss, but real-time and not 100% reliable. That fits for some
types of MPEG-4 content, but other types MUST be delivered with 100%
reliability, and I believe TCP is a better fit for these.

Colin

---

From: Dave Singer <[email-masked]>
Date: Fri, 17 Dec 1999 10:28:32 -0800

>information, and would be interested in arguments why not.

My (personal) interest in using RTCP-based re-transmission is to 
cover the case where you want the receiver clock to run, even if data 
never arrives, but you want the greatly improved (but not perfect) 
reliability.  You can't get both guaranteed delivery and on-time 
behavior.

If you are willing to wait indefinitely, I can promise to deliver 
everything (or at least you cannot prove that I won't).  For that 
case, TCP is mostly fine;  the only wrinkle is knowing when you are 
starving because the sender sent nothing, and when you are starving 
because the network is denying you data.

If you are not willing to wait indefinitely, TCP attempting for ever 
to re-transmit packets you no longer care about, and refusing to give 
you later packets that have already arrived, is unhelpful.  This is 
the space I'd like to see a RTCP-based algorithm fit into.

So I don't see it as a multicast/unicast thing, though of course an 
RTCP-based approach might lend itself to multicast.  However, you 
then have to assess questions like "should every recipient on a 
multicast signal loss of a packet, or should they watch each other?", 
"should I re-multicast this packet to everyone or re-unicast it to 
the complainers?", and so on.  Step 2, in my opinion.

David Singer
Apple Computer/QuickTime

---

From: Koichi Yano <[email-masked]>
Date: Fri, 17 Dec 1999 11:27:34 -0800

Colin Perkins wrote:
> >shouldn't be lost - imagine if the packet with all the decoding Hi,

Colin Perkins wrote:
> resilient to loss, but real-time and not 100% reliable. 

Basically, I agree with you.

But we can prepare the flag for strong request, which means "send
retransmission surely" wihtout thinking playout time or something.
I think it is a good idea.

By using a receiver's timer and tuning the frequency of re-request of
retransmission, reliability could be tuned.

Because the framework for retransmission request could be extensible to
ACK (not NACK), TCP-like 100% relible protocol could be implemented on
RTCP-rx.

I think just deployment of TCP is the easiest way for 100% reliability,
though.

Koichi

---

From: "M. Reha Civanlar" <[email-masked]>
Date: Fri, 17 Dec 1999 15:17:22 -0500

I agree that TCP is usable and sufficent for transferring the "vital" part
of the data in a real-time streaming application.

We actually used TCP successfully in transmitting high priority information with
MPEG2 ES transport. RFC2448 outlines our general approach and observations.

I believe, RTCP-Rx needs to be used when there is time for retransmission, but
the data to be retransmitted is loss resilient (i.e. would be useful if received
on-time but, not vital).

-----------------------------------------------------------------------
M. Reha Civanlar

---

From: [email-masked] (Alagu Periyannan)
Date: Fri, 17 Dec 1999 14:28:18 -0800

>Basically, I agree with you.

I think there is some merit in introducing the notion that certain RTP
packets are indispensible, i.e. 100% reliable. When such packets are lost
the receiver stops the clock and waits.

This RTP feature should be used only in non-interactive (i.e. streaming)
applications. It should not be used for video key frames and other such
error resilient packets.

We can keep saying use TCP for that, why bother enhancing RTP? If I use
TCP, I have to build a hybrid protocol that uses RTP/UDP and TCP to achieve
my goal. Why not make it simpler and offer the functionality in RTP?

This issue has been the cause of the growth of proprietary streaming
protocols by a number of vendors. Most of them have invented their own
protocol and do not use RTP because of this missing functionality. Even
some vendors such as Apple who are using RTP are implementing "repeat"
packets to get around this which is very inefficient.

Let's not preclude a "100% reliability" feature in RTP-Rx.

--

Alagu Periyannan                     [email-masked]
Entera, Inc.                         +1 510 770 5225

---

From: "M. Reha Civanlar" <[email-masked]>
Date: Fri, 17 Dec 1999 17:41:21 -0500

----- Original Message ----- 
From: Alagu Periyannan <[email-masked]>

> applications. It should not be used for video key frames and other such

I think, what the end point will do when a vital packet is lost is an
implementation issue. Sending a packet with 100% reliability, on the
other hand, is a protocol design issue. TCP does this for unicast and
there is a WG for reliable multicast.

> my goal. Why not make it simpler and offer the functionality in RTP?

The problem is that this won't make RTP simpler.

> packets to get around this which is very inefficient.

How can they achieve 100% reliability with "repeat" packets anyway?

---

From: Matt Podolsky <[email-masked]>
Date: Fri, 17 Dec 1999 16:25:52 -0800 (PST)

At 10:28 AM 12/17/99 -0800, Dave Singer wrote:
>My (personal) interest in using RTCP-based re-transmission is to cover the 
>case where you want the receiver clock to run, even if data never arrives, 
>but you want the greatly improved (but not perfect) reliability.  You 
>can't get both guaranteed delivery and on-time behavior.

I too agree with Colin and Dave's statements.  Our original vision of 
RTCP-Rx was a protocol that was generic enough that it could be applied to 
any RTP stream to improve its reliability, without making any 
guarantees.  We didn't want to have to write (or require others to write) a 
new specification or subtype for every payload out there.

However, as Koichi points out, we could add hooks to better enable a higher 
degree of reliability, such as something like a "retransmit this now, i 
don't care if you've sent it already or you think it won't arrive in time 
to be useful" flag on the receiver requests.  If one chose to one could 
then either write an MPEG-4-specific retransmission algorithm (as specified 
by the RXP subtype) or just write a receiver RTCP-Rx implementation that 
implements slightly different behavior if the payload is MPEG-4.  The 
MPEG-4 algorithm could then require critical information to be 
re-transmitted an unlimited number of times (with proper RTT spacing, of 
course) until it got to the receiver; if it wants to delay playback until 
it gets the critical decoding info perhaps it could couple this with a RTSP 
signal to pause the stream at the sender so the receiver's playback buffer 
doesn't get overrun.

In any case we wanted to avoid going into the specifics of implementing 
variable levels of reliability in the draft.  However, I'm not against 
providing hooks to enable these features, provided they don't cause things 
to become overly complex. And I think a "definitely retransmit" flag like 
the one I've described above would have additional value, specifically in 
distinguishing duplicate retransmission requests.  A receiver might repeat 
a retransmission request multiple times to protect against NACK loss, 
without wanting the sender to send more than one retransmission for the 
whole block of requests (this would be natural if it sent one NACK for each 
lost packet, since the BLP field allows up to 16 neighboring packets to be 
reported as lost).  At other times, the receiver might also repeat a 
retransmission request because the first retransmission was lost (or at 
least it thinks it was) and it has enough time before playout for a second 
attempt.  One way to distinguish between these two cases is to have the 
sender keep track of the time between a retransmission of a packet and a 
new request; if it's more than RTT+delta, it assumes the first 
retransmission failed and retransmits.  But as an alternative the receiver 
could instead simply set a timer when it makes the very  first request 
(possibly repeating it in subsequent NACKs); if the timer expires it sends 
a new request marked as "retransmit this now."  The sender in this case 
would just only send additional retransmissions after the first one if
the "retransmit now" flag was set (note that this means the re-requests can't
themselves be repeated for NACK-loss resilience, but that might be overkill 
anyway).

Still another idea would be to have a payload-specific algorithm (like for 
MPEG-4) in which the sender and receiver both know which pieces parts of 
the stream need full reliability; these would be ACKed instead of NACKed 
(by defining an ACK RXP subtype) and the sender would be responsible for 
reliability, a la TCP.

Matt

-- 

Dad always thought laughter was the best medicine, which I guess is why several
of us died of tuberculosis.

-- "Deep Thoughts" by Jack Handey

---

From: Dave Singer <[email-masked]>
Date: Fri, 17 Dec 1999 17:30:11 -0800

The only reason I might want to define a 100%-reliable mode in RTP is 
to get around the starvation puzzle I mentioned in a previous 
message:  I'm not sure if there is any way to tell if a TCP end-point 
is not giving you data because (a) it is waiting for a 
re-transmission which doesn't seem to be able to get through or (b) 
it can't get any response out of the source or (c) it is perfectly 
happy and the source isn't sending anything.

The problem with the clock-stop algorithm is that it basically says 
"stop the clock at the time for which you didn't get a packet that 
was sent";  of course, the time to stop the clock is in the packet 
you didn't get.  You can fix this, I guess, by stopping at the 
previous packet's time, or by indicating the next packet time in each 
packet (which also helps solve the starvation problem).
David Singer
Apple Computer/QuickTime

---

From: Colin Perkins <[email-masked]>
Date: Thu, 23 Dec 1999 14:11:56 +0000

>I think it is a good idea.

But not forgetting congestion control, right?

Colin

---

From: Colin Perkins <[email-masked]>
Date: Thu, 23 Dec 1999 14:10:30 +0000

>retransmission of these will hinder the real-time ability - so RTP/UDP/IP is

Of course - I think we all agree that TCP is not appropriate for streaming. 

>this in RTP/RTCP as far as I know. It would only be needed from client to

This is where the problem gets interesting - how to transport a mixed media
presentation? MPEG-4 is one example of this - but there are others - where
some parts of the presentation must be delivered reliably (there might be
some HTML and a Java applet, for example) and other parts can be streamed. 
My opinion is that this requires several protocols: it's not appropriate to
try to extend RTP so it can be used for all case.

There's a spectrum of reliability:

	reliable				unreliable
	not-timely				real-time
	+------------------------------------------------+
	TCP                         ~~~~~~~~~~~~~~~~~~ RTP

I envisage RTCP-Rx to be useful in the region labelled '~', for media which
is mostly real-time, but with a need for enhanced reliability subject to
some time bounds. I don't see it as appropriate for fully reliable streams.

Sorry for keeping on about this - but I think it's important to scope the
problem we're trying to solve with RTCP-Rx, before we get into the details
of the protocol. 

Colin

---

From: Colin Perkins <[email-masked]>
Date: Thu, 23 Dec 1999 13:53:05 +0000

>Let's not preclude a "100% reliability" feature in RTP-Rx.

I think "100% reliable" is a very loaded phrase. I accept that we may need
to allow for streaming media, where some packets have to be delivered with
very high probability, but with some time bound. This is where I believe
RTCP-Rx fits in.

However, I don't believe we want to use RTCP-Rx for streams where the
entire stream is to be delivered 100% reliably. That's not streaming
anymore, and is the case where TCP is used.

I think this means I agree with you - but we should be careful when we say
we want 100% reliable transmission using RTCP-Rx.

Colin

---

From: Thomas Marshall Eubanks <[email-masked]>
Date: Thu, 23 Dec 1999 13:46:58 -0500

> try to extend RTP so it can be used for all case.

Hello;

     Any real time system has to acknowledge the possibility of
drop-outs and data loss. The goal has to be to set specs on what is
acceptable and to engineer things to meet that. If you are saying that certain data
cannot, no matter what, be lost, then you are not talking about real time transport,

because at some times preventing that will require retransmission.  This is
really a function of reliable multicast transport (RMT - [email-masked]).
Otherwise this is coming close to re-inventing a (not yet fully invented) wheel.

If you are really operating in real-time, and you say that some bits are more
important than others,
then what you are really saying is that some bits need more redundancy than others.
For example, if there was low rate text (say html, which does not gracefully suffer
dropouts) combined with some high rate audio or video stream (which does), the low
rate
text could be sent in some highly redundant fashion, so that it could be
reconstructed
if pieces were lost.

The question I would have is, should this be allowed for at the protocol level,
or should this be viewed as an application level problem that the protocol should
avoid ?
I would argue that the most sensible way of doing this in a protocol would be to
allow
for sub-streams, which could have different levels of redundancy.

Regards

Marshall Eubanks

T.M. Eubanks
Multicast Technologies, LLC
e-mail : [email-masked]

---

From: Colin Perkins <[email-masked]>
Date: Thu, 23 Dec 1999 20:21:14 +0000

--> Thomas Marshall Eubanks writes:
>acceptable and to engineer things to meet that. If you are saying that
>certain data cannot, no matter what, be lost, then you are not talking
>about real time transport, because at some times preventing that will
>require retransmission.  This is really a function of reliable multicast
>transport (RMT - [email-masked]).  Otherwise this is coming close to
>re-inventing a (not yet fully invented) wheel.
>
>If you are really operating in real-time, and you say that some bits are
>more important than others, then what you are really saying is that some
>bits need more redundancy than others.  For example, if there was low rate
>text (say html, which does not gracefully suffer dropouts) combined with
>some high rate audio or video stream (which does), the low rate text could
>be sent in some highly redundant fashion, so that it could be
>reconstructed if pieces were lost.
>
>The question I would have is, should this be allowed for at the protocol
>level, or should this be viewed as an application level problem that the
>protocol should avoid ?  I would argue that the most sensible way of doing
>this in a protocol would be to allow for sub-streams, which could have
>different levels of redundancy.

We have traditionally sent each media type as a separate RTP stream, so it
is simple to provide different levels of protection for each. This is one
of the main reasons why multiplexing media streams together in RTP is not
always a good idea.

Colin

---

From: [email-masked] (Alagu Periyannan)
Date: Thu, 23 Dec 1999 17:57:11 -0800

>The problem is that this won't make RTP simpler.

Correct. But if we take the pain and introduce this functionality in RTP-Rx
it will provide a standards-based way to deliver streams in which a few of
the packets require 100% reliability. Currently the only way to do this is
inventing my own protocol.

>How can they achieve 100% reliability with "repeat" packets anyway?

That's right you can't achieve 100% reliability with this method. I was
trying to illustrate how vendors are trying solve the reliability issue
within the current RTP framework.

--

---

From: [email-masked] (Alagu Periyannan)
Date: Thu, 23 Dec 1999 18:17:49 -0800

>we want 100% reliable transmission using RTCP-Rx.

Actually the fundamental debate here is whether RTP is solely a "realtime"
protocol or is it a "time-based streaming" protocol.

A "realtime" protocol in my definition is a protocol that delivers
timestamped packets in a time bounded fashion. Since it needs to deliver
packets in a time bounded fashion it can theoretically never achieve 100%
reliability.

A "time-based streaming" protocol in my definition is a protocol that
delivers timestamped packets period. It must cover the case of time bounded
delivery with no reliability guarantees and the case of 100% reliability
with no time bound.

A time-based streaming protocol's functionality is a superset of a realtime
protocol's functionality. I would like to see an IETF standard for a
time-based streaming protocol. The question is do we build that off RTP
(and its popularity and success) or do we just invent yet another RTXYZP
protocol?

--

---

From: Colin Perkins <[email-masked]>
Date: Mon, 27 Dec 1999 13:56:36 +0000

>delivery with no reliability guarantees and the case of 100% reliability

We have RTP over UDP for the first case, and the new profile draft specifies a 
TCP encapsulation for RTP which should deal with the case of "100% reliability
with no time bound"? 

This leaves the case where we have a time bound, but desire enhanced reliability 
compared to UDP. We have some options here already (eg: redundant audio and
parity FEC), and the RTCP retransmission draft offers another alternative.

Colin

---

From: [email-masked] (Alagu Periyannan)
Date: Tue, 28 Dec 1999 11:31:31 -0800

>parity FEC), and the RTCP retransmission draft offers another alternative.

I was not aware of the RTP over TCP draft. (A pointer is appreciated.)

Nevertheless, in many cases one may need to mix the various cases I
mentioned in one RTP session. For example if we are streaming an animation
over RTP, we may want to send the "actor" information in a reliable fashion
and the motion vectors in a non-reliable fashion.

This is not possible if the "100% reliability with no time bound" case is
covered in a separate profile that does not include features of the
standard A/V profile.

--