Subject: [MMUSIC] RE: [dccp] natural DCCP sender-side API: (was stream switching)

From: "Phelan, Tom" <[email-masked]>
Date: Thu, 24 Jul 2003 10:08:20 -0400

Hi Phillipe,

From your point below - I don't see much difference between this (one-way
pre-recorded media) and file download either.  Just that in "normal" file
download it wouldn't be reasonable to consider changing the encoding scheme
in mid-transfer in order to keep up with some real-time considerations.

Note that to simplify things, my discussion assumed that the receiver could
buffer the entire program if it needed to.  I think it would be pretty easy
to extend the scheme to handle receivers that couldn't buffer the whole
program.  Something like a fourth buffer threshold that if hit causes the
sender to pause for a short time ...

Tom P.

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]]
> Sent: Wednesday, July 23, 2003 1:43 PM
> To: Phelan, Tom
> Cc: [email-masked]; [email-masked]
> Subject: RE: [dccp] natural DCCP sender-side API: (was stream 
> switching)
> 
> 
> Hi Tom,
> 
> you write:
> 
> >The relevant difference, as far as this discussion goes, 
> between recorded
> >and live media is that recorded media can be transmitted 
> from the sender as
> >fast as the network allows (assuming buffering at the receiver).
> 
> and then latter (still about recorded media):
> 
> >So, in this scheme, the sender doesn't need to know the rate 
> that TFRC has
> >determined, it simply always sends as fast as TFRC allows.
> 
> 1) I am sorry but *from a network point of view*
> I dont see a difference between this and "download"?
> 
> Except that to my knowledge all the products doing that use HTTP
> since it solves a rather large number of other problems at 
> the same time!
> 
> 2) Now *from a user point of view*
> 
> "download" has a flavor usually called "progressive download"
> whereby the downloaded media file has a structure enabling
> to start playback before download is finished.
> 
> In this mode usually the slide bar that the player
> uses to display the media time has 2 colors,
> one displaying the current time in the media,
> the other the *received* time in the media,
> if this second mark stays ahead of the first mark playback is 
> un-interrupted,
> otherwise the player stops and displays a message such as 
> "please wait, rebufferring".
> 
> So I dont see either, from a user point of view,
> how different a behavior the case you describe would have?
> 
> ****************
> 
> I conclude about this use case of the DCCP blocking API you described,
> that it is not very attractive at all.
> 
> Surely DCCP has better assets to lure implementers?-)...
> I think an API providing a target rate would be a good 
> starting point;-)
> 
> regards,
> 
> Philippe Gentric
> Software Architect
> Philips MP4Net
> [email-masked]
> http://www.platform4.philips.com
> 
> 

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: "Rosen, Brian" <[email-masked]>
Date: Thu, 24 Jul 2003 11:35:05 -0400

As someone who is presently making such a videophone, let me add just
a couple of details, as I think Mark's observations are very close
to what we need.

Most video codecs cannot quickly change their encoding rates, and they
often encounter a glitch when they do.  Many are based on hardware
implementations in the encoder which force you to stop the encoder, change
a bunch of settings, and restart it.  It's possible, and we'll do it,
but not very often, please.

Second, we have found that delay is not just a problem, it is a cliff.
If you go over 150 ms or so, you fall off the cliff.  The difference 
between, say 250 ms and 500 ms is about 2 orders of magnitude less
that the difference between 150 ms and 200 ms (there is in fact some
variation, because it's a human response thing, 150 ms is something like
a 95% point for detecting delay).  These are one way delays, and,
as an aside, are actually a limit for the audio, but are determined
by the video, because you need the audio to match the video (lip sync)
the video delay is harder to control (codec complexity), but the
limit is your ability to hear delay.  So, generally, we have NO
buffering (like less than 1ms) for any network related things.
On the other hand, to make it work, we, unlike many, have relatively
constant data rates out of the encoder without using buffering to
accomplish it.

So, we can tolerate a reasonable bound on TRFC rate variation, we'll
always send at that rate, or less.   In practice, you tend to
quantize transmit rates, so we will have to send at the next lowest
rate for a given TRFC limit.   Some other codecs can be forced to
an exact data rate by buffering peaks, and again, you often see
quantization of rates.

A small problem that arises is that the signalling (SIP) typically
includes a negotiation of rates.  There isn't a way to signal a 
dynamically changing rate.  This is straightforward to address by 
us sip guys.  If it changes slowly enough, we can just signal
the change using the mechanisms we have.

In VoIP, things are a little more complex.  There are a variety of
algorithms, and a few of them are defined to work on more than one
data rate.  In general, like the hardware encoder, you can't switch
algorithms without a noticeable glitch.  Some implementations of
multirate codecs can switch reasonably smoothly, others can't.
It's presently uncommon to find a VoIP phone with a variable data
rate codec, but it is common to find VoIP phones with more than
one algorithm, each algorithm having a different data rate.  Switching
algorithms is definitely a signalling problem.  On the other hand, 
nearly all voice codecs are constant bit rate inherently.

Brian

> From: Mark Handley [mailto:[email-masked]]
> Sent: Thursday, July 24, 2003 10:46 AM
> To: Phelan, Tom
> Cc: 'Damon Lanphear'; [email-masked]'; Magnus 
> Westerlund;
> [email-masked]; [email-masked]; Pasco Anderson, Jay
> Subject: Re: [dccp] natural DCCP sender-side API: (was stream 
> switching)
> 
> 
> 
> 
> >Actually, my big worry here is TFRC, not the API.  I'm 
> baffled about how to
> >map slow start plus mostly smooth changes but occasional 
> abrupt changes and
> >a small playout buffer into something workable.
> >
> >Another thing that concerns me (touched on in the paragraph 
> above) is the
> >disconnect between a transport protocol making rate decisions and an
> >application making rate decisions.  It seems to me that 
> these particular
> >applications are much better equipped to make the rate 
> decisions - they know
> >about what is possible.  It also seems that the application 
> can make at
> >least as good a decision about what constitutes network 
> congestion.  The app
> >will need feedback about the playout buffer state, and this 
> is largely
> >redundant with the DCCP acks.  If the app is making rate 
> decisions and
> >detecting network congestion, what do we need DCCP for?
> 
> [I'm just dropping back into a discussion I haven't been following, so
> forgive me if this duplicates previous discussion]
> 
> There seem to be two obvious sorts of apps that might use TFRC:
> 
>  - streaming media
>  - videotelephony
> 
> That isn't to say that there are different apps, but these two typify
> two classes of apps.  Both are fine with variable packet rates.  But
> streaming media has significantly more playout buffer available.
> 
> Streaming media apps can run the codec (or choice of pre-encoded
> stream) at approximately constant bitrate, and send at variable
> bitrate, using the playout buffer to deal with the temporary
> mismatches.  
> 
> Videotelephony apps don't have sufficient delay budget to fix things
> at the receiver, but they do have the luxury of encoding live, so they
> can change the compression parameters on the fly.  They usually don't
> have luxury of re-encoding twice though, so if their choice of
> quantization parameters gives a rate that is different from what is
> expected (because the video content was unpredictable), then they
> might be in trouble.  Also, if the TFRC bandwidth envelope changes
> unexpectedly, you can be in trouble.
> 
> 
> So, why do I say all this?
> 
> 
> My assumption is that streaming media apps will normally send at the
> TFRC rate.  What they need is sufficient advance warning that the
> playout buffer is running low so that they can switch down to a lower
> rate.  And if the playout buffer is starting to get rather large, then
> they can switch up to a higher rate.  Assuming rate switching is not a
> trivial operation, and that the rate can only be switched at certain
> re-sync points, then there is some danger that if congestion suddently
> gets really bad, then the playout buffer will be exhausted before the
> codec can be switched.  My assumption here is that a really good API
> would allow the sender to send above the TFRC rate for a period of
> time, so long as the rate mismatch isn't too bad.  A factor of two
> might be reasonable as an upper bound on the oversending rate.
> 
> 
> Videotelephony apps probably can't send at precisely the TFRC rate
> without sacrificing delay to an undesirable extent.  This means two
> things. First, the app needs to be allowed to send at rates above the
> TFRC rate for a period of time (again, bounded to avoid extreme
> unfairness).  Second, if the app sends under the TFRC rate for a short
> period, it shouldn't lose it's ability to send faster later, so long
> as it didn't undersend by too much.  For such apps, the TFRC rate is a
> guide rate - the app should be capable of varying around the guide
> rate, so long as it stays within reasonable bounds, and so long as the
> long-term average is close to the TFRC rate.
> 
> 
> Anyway, in both cases, if the kernel strictly enforces the TFRC rate,
> app designers will either not use DCCP, or they'll pad their signal
> with redundant data so they can operate within a more predictable
> codec envelope.  Both are undesirable.
> 
> 
> I'm not sure how we best design an API to satisfy this, but I think
> it's really important that we do think about how this can be achieved.
> 
> 
> The general philosophy that the TFRC rate is a guide-rate to be
> followed loosely will probably also apply with TFRC-PS and VoIP, but
> we know a lot less about this space.
> 
> Cheers,
> 	Mark
> 
> dccp IETF mailing list: [email-masked]
> list info:  https://www1.ietf.org/mailman/listinfo/dccp
> wg charter: http://www.ietf.org/html.charters/dccp-charter.html
>

---

From: [email-masked]
Date: Thu, 24 Jul 2003 20:06:44 +0200

Hi Brian,

>we'll always send at that [TFRC] rate, or less.   In practice, you tend to
>rate for a given TRFC limit.

I am glad to see that we reach exactly the same conclusion ...

it is funny that it seems to be for reasons that look so different !

regards,

Philippe Gentric
Software Architect
Philips MP4Net

                                                                                                                                       
                                                   To:   "'Mark Handley'" <[email-masked]>                                     
                                                    "Phelan, Tom" <[email-masked]>                                               
                                                   cc:   "'Damon Lanphear'" <[email-masked]>                                        
                                                    Philippe Gentric/MP4-SUR/CE/PHILIPS@EMEA1                                          
               "Rosen, Brian"                       Magnus Westerlund <[email-masked]>                                 
               <[email-masked]             [email-masked]                                                                      
               >                                    [email-masked]                                                                    
                                                    "Pasco Anderson, Jay" <[email-masked]>                                    
               Sent by:                            Subject:    RE: [dccp] natural DCCP sender-side API: (was stream switching)         
               [email-masked]                                                                                                     
                                                   Classification:                                                                     
               24/07/2003 17:35                                                                                                        

Brian

---

From: [email-masked]
Date: Fri, 25 Jul 2003 15:02:50 +0200

Damon,

>> yes on advance warning about the TFRC rate,
>
>How can you ever get an advance warning about TFRC rate changes.  These
>changes are response to immediate stimuli, to my knowledge that are not
>predicted by the current suite of algorithms.

Yes.

TFRC can update the rate computation everytime it receives feedback from the receiver.

What I would like is that the sender application gets this information as soon as it is updated.
this is the meaning of "advance warning" for me.

I can use it to drive my media source (RT encoder or stream switch).

It contrast sharply with a "blocking write" which is "too late" for me:
once my source has produced a frame what I am supposed to do
if DCCP blocks it ? garbage it ? wait? but how long should I wait?
wait for what? for a timeout ? but what will happen after the timeout ?
will I write this frame or not ? If I have to change the rate how am
going to know which rate to choose ? etc ...

on the other hand I would expect that DCCP never blocks a write
if the source always follows the DCCP indicated rate ....

regards,

Philippe Gentric
Software Architect
Philips MP4Net

---

From: "Phelan, Tom" <[email-masked]>
Date: Fri, 25 Jul 2003 09:39:05 -0400

Hi Eddie,

See inline ...

Tom P.

> From: Eddie Kohler [mailto:[email-masked]]
> Sent: Friday, July 25, 2003 2:50 AM
> To: Phelan, Tom
> > If DCCP conflicts with an application's requirements, then 
> the application
> > won't use it, whether DCCP is network friendly or not. 
> 
> Totally agree. And totally agree that we must figure out 
> whether DCCP is
> really useful for two-way voice. But, for example, an 
> application with a
> requirement "must achieve a minimum bandwidth" would not be 
> appropriate for
> DCCP, because it requires non-TCP-friendly congestion control.

I think I'd rephrase that as "must achieve a minimum bandwidth and everyone
else be damned would not be appropriate".

> Nothing that you have discussed is that far out, but some of 
> it is getting
> far out enough that I wanted to make the concern explicit. 
> (For instance,
> we've been talking about apps that can transmit only at a 
> discrete set of
> rates---unless 0pps was one of those rates, such an app would have a
> minimum bandwidth requirement.)

I haven't mentioned this in my e-mails so far because they were long enough
without it, but I assume that when the TFRC rate falls below the minimum
coding rate we throw away packets to shape the flow to the TFRC rate.  As I
said before, it seems better to me that the sender makes intelligent
decisions about packet discard, rather than the network makes random
decisions.  Eventually you could get to the point that the media stream is
completely unintelligible, and maybe you should disconnect.  I certainly
don't feel that you should keep blasting away at some minimum acceptable
rate regardless of what happens to those packets in the network.

> 
> > Note that I think
> > that network-friendly reaction to congestion is absolutely one of my
> > application's requirements (my app is two-way voice).  I 
> hope that I can use
> > DCCP to get that, but I'm having difficulties making it 
> work.  Not that I've
> > given up hope, mind you.  I think I've managed to work out 
> reasonable ways
> > to use DCCP for similar, but not quite identical apps.  And 
> I intend to keep
> > trying for my app.
> > 
> > I think it's very important for DCCP to know whether it's 
> really useful for
> > two-way voice or not.  All of the DCCP documents mention it 
> as a primary
> > target application.  Some people on this list might already 
> know how to make
> > it work, but I don't yet, and I haven't seen anyone give a 
> credible scheme
> > for it yet, so I'll keep trying.
> 
> FYI I really appreciate this discussion!!!
> 
> Eddie
>

---

From: "Rosen, Brian" <[email-masked]>
Date: Fri, 25 Jul 2003 10:16:43 -0400

The requirement is pretty easy to state:

For two way voice, the data rate is quantized, and constant.  You can
tolerate some packet loss, which is in the percent or so range without
problems, and some packets are more important than others.  It takes
some time to change rates.  Delay is one of the most important parameters.
Add any significant delay, and we can't use it.

For video, the data rate can be less constant (ours is only slightly
variable), there are more reasons to do sender marking of packets to
be discarded, and delay is even more important.

A nice characteristic you can use is that voice and video has the
possibility of admission control.  You can say that a transport
protocol can't really directly deal with that, but at least at the
moment, that's how you get a rate change, so the same mechanism
can get you admission control.  In the end, you either get an
acceptable rate that you can send constant bit rates at, or you
stop sending at all.  

I'm more sanguine about complexity.  I'll tolerate some level of code
complexity if I get something out of it.  Let's face it, TCP-friendly
isn't something that matters to me.  Better experience for our
users is.  If TCP-friendly ultimately results in a better user
experience, good.  If it doesn't ....

Brian

> Subject: [MMUSIC] Re: [dccp] natural DCCP sender-side API: (was stream
> switching)
> 
> 
> Hi Tom,
>

---

From: [email-masked]
Date: Fri, 25 Jul 2003 17:39:15 +0200

Mark,

>>What will happen if an application "does not use" the
>>bandwidth indicated by TFRC?
>
>OK, my assumption here was that streaming media was coming from disk.
>Thus you can always send faster then the receiver is consuming.

I am not sure about what you mean by "always" but what I am sure is that it cannot be "all the time":
specifically you can do that up to the moment when your client side buffer is full,
because a media decoder is not regulated on a *data* rate but on *media time*...
so if you send data faster than the media rate you fill up the client buffer ...

(which brings back the previous conclusion that pre-recorded media is not that much different than one-way "live")

regards,

Philippe Gentric
Software Architect
Philips MP4Net