Subject: Re: [MMUSIC] stream switching

From: [email-masked]
Date: Mon, 14 Jul 2003 12:02:36 +0200

Magnus, you wrote:

>Hi Philippe,

   Hi Magnus, thank you for the feedback! It will definetely help
   me in the job of improving this draft.

>
>I have now had time to read your requirements draft and has some
>comments about this. I also have been rather heavily involved in the
>work within 3GPP to define bit-rate and content-rate adaptation so I
>have some comments based on this work.
>
>First, the goal of the stream switching work is in my opinion to:
>- Enable the receiver to receive interrupt free media at highest
>possible quality.
>

   sure, if this is not obvious from the "requirement" draft then I
   need to reword the introduction!

>To enable this the complete system must ensure the following things:
>- Congestion Control or rather adapting transmission rates to available
>network rate => avoiding high packet loss rates that severely effect
>quality.

   yes

>- Avoid that the receivers buffers either under flow (causes interrupt)
>or overflow (show up as packet loss).
>

   yes, this is fundamental for seamless-ness

>Based on this one can arrive at the conclusion that the media or content
>rate must be adapted, otherwise the underflow condition will be
>impossible to avoid. If one performs content adaptation towards lower
>bit-rates one must also perform adaptation towards higher rates.
>

   I am not sure about this "must", could it be that your are
   considering differently media rate and transmission rate ?

   because I am not considering changing the transmission rate (as a
   end-to-end application issue) => stream switching is only about
   changing the *media* rate.

   what I want to mean is you dont *need* to increase the media rate
   to avoid undeflows but instead you may *want* to increase the
   media rate in order to improve the user experience in terms of
   image quality...

>This I think is the basic requirement and assumptions that apply to your
>requirements draft.
>
>Based on this any solution can be divided up into different
>functionality blocks:
>
>- Transmission rate control, this functionality ensures that a sender
>does not send more data then the network can transport without being to
>unfair, or overflow network buffers.
>

   hum, again i prefer the term "media rate control" if you dont mind?

>- Receiver buffer control, this functionality ensure that one does not
>overflow the client's receiver buffers.
>

   yes, not sure there is much to "do" there in terms of IETF spec?

>- Content rate control, functionality to ensure that the transmitted
>date fits the available network rate, while at the same time ensure that
>the receiver buffer does not underflow.
>

   yes! this is stream switching principle, but again it is about
   "media rate", not changing the transmission rate.

>Based on this functionality blocks one can use one or several
>protocols/solutions to realize them. They do not need to be separate and
>can utilize information enabled by any of the other functionality
>blocks. However I believe that all parts must be present to have a
>complete solution. Also how to realize the solutions is a not fixed by
>this.
>
>I would suggest that one tries to group the requirements regarding which
>of the functionalities they address.
>

   Which groups would you suggest ?

>When one comes to solution one must start discussing what control
>paradigm one desires, as it will heavily influence the solution. Three
>different ones are:
>- Server controlled: In this the server is the responsible for all
>decisions regarding the adaptation. The client simply reports certain
>measurements that the server interprets.
>
>- Client controlled: The client makes the decisions and interpretations
>of what it see. Then it issues commands to the server that it must
>follow. The server is a simple slave.
>

   slave, yes, but a slave that *could* refuse to obey ;-) e.g. as
   discussed in my draft for rate control a server must be allowed to
   deny a request for a higher data rate...

>- Hybrid control: In a hybrid control scenario both client and servers
>makes decisions and interpretations of the data.
>
>I will not try to compare these solutions this if for future
>discussions. If one is interested in a server controlled solution one
>can look at the latest proposal in 3GPP SA4:
>http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_27/Docs/S4-030501.zip
>Look at the buffer reporting mode variant. This proposal provides
>receiver buffer control and content rate control. The transmission rate
>control is handled through RTCP reports with enabled extended reports of
>packet loss vectors.
>

   I have to say that using such RTCP extensions looks very much like
   "client control" to me ;-) but I dont really care about the
   terminology ;-)

>
>Some more direct comments on the draft:
>
>1. Introduction: The encoding could be done in real time allowing
>maximum flexibility in the produced content rate. However this is may
>not result in optimal encodings and are also very resource demanding for
>a server.
>

   yes, I did not mention this possibility because in large scale
   media distribution this would effectively require one "production"
   encoder per client, which is -even in several years from now- way
   beyond the cost points for this type of products.

>2. Section 2.1.1 and 2.1.2: I think the behaviour in this case should
>really be depending on the operators policies. I would not like to have
>my expensively paid on demand video reduced in quality if this network
>is of the type you describe. However I can see that we should enable this.

   Yes, you are right, all this assumes the correct commercial
   agreements are in place. Our experience is however that a end user
   always prefers a continuous playback, even a degraded one,
   providing that the degradation is "resonable", all other solutions
   including "freezes" for rebuffereing are not acceptable for the
   wide public who is accustomed to TV ;-(

From an IETF point of view (hey, just giving my own opinion ;-)
I think it would be a good idea that implementations *by default*
would "behave a la TFRC" rather than what we have today,
because otherwise -as sally floyd pointed out this morning-
we would run into rather nasty trouble.

>
>3. Section 2.2: I think you need to be a bit more clear that this is one
>example of a service. Also the AMR rate of 5 kbps is a bit to low. The
>4.75 kbps mode has a quality that is not the best. However to achieve
>this bit-rate you must only send one AMR packet every 1.28 seconds,
>which means 64 amr frames per packet. This is not realistic as every
>packet will be required to reach the receiver to achieve acceptable
>quality. I would think that reasonable AMR qualities are using at least
>10 packet every second, meaning that a header overhead of 3 kbps.
>

   right.

>4. Section 2.5: I think this section assumes something that are not
>quite true. First the impossibility to signal this rate switch using
>current protocols.

   What *is* impossible is to find a comprehensive spec for stream
   switching !-) and it is my intention to provide one, i would love to
   "use the current protocols"!

>I claim it is possible but most likely not supported
>by any available client.

   And that would be the point in writing that spec...

>To signal the rate switch in SDP you present
>one different RTP payload types for each codec and configuration, most
>likely at least 4 maybe 5. The 40 and 50 kbps AAC stereo may be possible
>to encode using the same AAC configuration. The declared payload types
>are present in a SDP. A client shall not accept the session and setup
>the media stream unless it can handle all these codecs. If it has setup
>this media stream it has in fact accepted that any of the codec may be
>used in the media stream.
>

   well, I completely agree that *that* may be a solution to stream
   switching, the point then would be to document it!-)

>The server can here use different strategies when switching. The most
>likely one is to use a single SSRC and in the same RTP sequence number
>and timestamp space send the selected media quality.

   agreed, this is a very nice possibility...that obvioulsy we would
   need to *seriouly* document ... because this is not the "default"
   thing at all from existing specs, right?

>When the server
>switched this will be indicated by the new RTP PT. This has a slight
>problem as RTP/RTCP does not handle switched RTP TS rate perfectly. The
>client may in fact lose synchronization until it receives a RTCP packet
>after the switch has taken place.

   right! that is a key issue!

>This happens if the packet where the
>RTP TS rate switches is lost. This is not a problem as long as the RTP
>TS rate before and after the payload type switch has the same TS rate.
>In other words not a problem for any video codec which all runs at
>90000.

   yes, not a problem *if* (as mentionned above) you mandate the same
   RTPTS space for all streams ...

>This is more a problem for audio codecs. I would in fact try to
>address this problem with audio codecs by ensuring that any audio codec
>is allowed to select a RTP TS rate that is a even rate multiple of its
>native value. This would allow audio codecs to run at a the smallest
>common denominator rate.
>

   Is it always possible/efficient?

>The other strategy at points of switching would be to use different
>SSRCs. This would violate the semantics that a SSRC represents a certain
>source. This also has some problems with the synchronization of the
>different SSRCs. This can't be done in RTSP as RTSP due to an erroneous
>definition of RTP-Info did not map each synchronization information to
>the SSRC. This can only be fixed through a new header due to backwards
>compatibility problems. Otherwise RTCP could be used. By sending SR for
>all SSRC one could in fact ensure that the synchronization is available.
>It will utilize more RTCP bandwidth.
>

Hum, so we should disregard this option?

>5. Section 3.1.2: I don't think you should mix congestion control with
>adaptation control that hard. In my opinion, how drastic a content rate
>change is needed and how much depends on the clients buffer status.
>

sorry, I do not understand that one?

>6. Section 3.2.1: I think this directly falls under Transmission rate
>control and is a function of the congestion control mechanism.
>
>7. Section 3.2.2.1: I don't think that this is a valid requirement. This
>is clearly an option that must be available. The controlling entity can
>trade off sending new data against retransmitting lost data. This trade
>off needs to be available to maximize quality. Further this can be done
>without causing congestion as long as the sender keeps to the available
>network rate. This is why splitting the functionality into different
>functionality blocks are so important. By having this model one can see
>that a server can have network rate N(t), content rate C(t) and
>retransmission rate R(t). The controlling entity must ensure that
>C(t)+R(t) <= N(t). Further the clients buffer allows that on short time
>scales R(t) can be equal to N(t) as long as enough data are available in
>the clients buffer.

   fully aggreed, the point was to explain that FEC increases the
   data rate (which a lot of people seem not to be aware of !-) but I
   agree recommending "not using FEC" is too blunt

>
>8. Section 3.7: I don't think we can deploy a solution for switching
>that gives full functionality and at the same time is backwards
>compatible. However think on how one can maintain some of the
>functionality and being backwards compatible may be a worth the effort.
>However this is rather a design evaluation criteria than a requirement.
>

   do you mean it is not appropriate to have backward compatibility as
   a requirement?

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.platform4.philips.com

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 14 Jul 2003 14:40:20 +0200

Hi Phillipe,

I hope that I can clarify anything that was unclear. I also has some 
further comments, see below.

>   considering differently media rate and transmission rate ?

What I mean is that if the available transmission rate is less the media 
rate then you on some time scale if nothing changes you will empty your 
buffer. Therefore you "must" change the media rate to avoid this or have 
interrupts. I don't know about any other solution.

>   changing the *media* rate.

I don't think you can exclude the transmission rate from the equation. 
In my opinion you need to consider transmission rate, receiver buffers, 
and the media rate all together.

>   media rate in order to improve the user experience in terms of
I am not sure what you mean, but I guess the following. In cases you are 
not transmission rate limited one is not required to increase the media 
rate, however it is commonly desired. I can agree that there are some 
need to allow the receiver to specify what is allowed in respect to 
media rate modifications.

>   hum, again i prefer the term "media rate control" if you dont mind?

No, I don't think you completely understand. In a system where the 
buffer space is known you don't have to transmit the data in the same 
rate as the media rate. Due to the buffer one can during some parts of 
the session transmit data at rates that are both higher and lower than 
the media rate. However when doing this you need to ensure that you are 
not under or overflowing the receiver buffer. What you try to do is to 
simplify this system to basically say that the transmission rate is 
equal to media rate which I think prevents better performance.

If any one does not understand this, please prod me and I will try to 
provide a better explanation of the complete system.

>   yes, not sure there is much to "do" there in terms of IETF spec?

There is nothing in IETF that allows feedback or even specification 
about the receiver buffers to or by the sending side. This is analogous 
to TCP's receiver window that allow to control the flow of media.

>   "media rate", not changing the transmission rate. 

This I can agree is what you describe as media rate control. By 
selecting or controlling the media rate one can meet the requirements of 
transmission rate and receiver buffers.

>>of the functionality's they address.
>>
>>    
>>
>

Some suggestions would be:

Media requirements: Requirements to prevent the media from being 
corrupted or worsen quality, like limited packet losses, gap free 
switching, RAP,etc.

Signalling requirements: Requirements that are a related to what the 
signalling mechanism needs to achieve.

System requirements: Requirements that have more general feeling, like 
backwards compatibility, etc.

>   deny a request for a higher data rate...
In a true client control paradigm, this may not even be true. But the 
basic point is that the requirements may motivate the choice, but we 
should not choice a a paradigm and then write the requirements to fulfil 
them. I have a rather long list of comments to when this discussion 
needs to be started.

>   "client control" to me ;-) but I dont really care about the

Actually the mechanism in my opinion is very server controlled. The 
client provides the server side with some parameters telling it what to 
achieve. Then the client provides regular feedback to allow the server 
to run the algorithms and control the whole system.

>   beyond the cost points for this type of products.
I think that spending two sentences on this is useful, just to make it 
clear that it has been considered but has significant problems.

>we would run into rather nasty trouble.

Yes, I agree that over any best effort network you need some type of 
congestion avoidance. So default behaviour is good, however we must also 
allow for systems that has QoS agreements.

>   switching, the point then would be to document it!-)

Good, then you should rather document this deficiency rather than 
claiming that it is possible.

>   thing at all from existing specs, right?

I have seen it as being the default behaviour, however this is nothing 
that has been thoroughly discussed in AVT and the RTP spec not be that 
clear on this point. Documenting this would be a good thing.

>   yes, not a problem *if* (as mentioned above) you mandate the same

No, not for the currently specified RTP payload formats. It is something 
I will recommend in future payload formats to allow for this functionality.

>Hum, so we should disregard this option?

I think so.

>sorry, I do not understand that one?

This has to do with the complete system that a sender can transmit at 
one rate, while the media rate has another rate. The discrepancy is 
handled by the receiver buffer. Therefore the control of the media rate 
can be performed more drastic or not depending on what the buffer status 
is. An almost empty receiver buffer needs to be handled quickly and 
drastically, probably be reducing the media rate significantly. If the 
buffer are rather full the handling of this can be much less drastic, a 
small discrepancy can simply be tried to waited out. Or the quality 
change can be minimal.

>   do you mean it is not appropriate to have backward compatibility as

It might, however I think it should be further considered. I think 
listing the level as critical is to high level.

Cheers

Magnus Westerlund