Subject: Re: [MMUSIC] stream switching

From: Magnus Westerlund <[email-masked]>
Date: Fri, 11 Jul 2003 18:11:04 +0200

Hi Philippe,

I have now had time to read your requirements draft and has some 
comments about this. I also have been rather heavily involved in the 
work within 3GPP to define bit-rate and content-rate adaptation so I 
have some comments based on this work.

First, the goal of the stream switching work is in my opinion to:
- Enable the receiver to receive interrupt free media at highest 
possible quality.

To enable this the complete system must ensure the following things:
- Congestion Control or rather adapting transmission rates to available 
network rate => avoiding high packet loss rates that severely effect 
quality.
- Avoid that the receivers buffers either under flow (causes interrupt) 
or overflow (show up as packet loss).

Based on this one can arrive at the conclusion that the media or content 
rate must be adapted, otherwise the underflow condition will be 
impossible to avoid. If one performs content adaptation towards lower 
bit-rates one must also perform adaptation towards higher rates.

This I think is the basic requirement and assumptions that apply to your 
requirements draft.

Based on this any solution can be divided up into different 
functionality blocks:

- Transmission rate control, this functionality ensures that a sender 
does not send more data then the network can transport without being to 
unfair, or overflow network buffers.

- Receiver buffer control, this functionality ensure that one does not 
overflow the client's receiver buffers.

- Content rate control, functionality to ensure that the transmitted 
date fits the available network rate, while at the same time ensure that 
the receiver buffer does not underflow.

Based on this functionality blocks one can use one or several 
protocols/solutions to realize them. They do not need to be separate and 
can utilize information enabled by any of the other functionality 
blocks. However I believe that all parts must be present to have a 
complete solution. Also how to realize the solutions is a not fixed by 
this.

I would suggest that one tries to group the requirements regarding which 
of the functionalities they address.

When one comes to solution one must start discussing what control 
paradigm one desires, as it will heavily influence the solution. Three 
different ones are:
- Server controlled: In this the server is the responsible for all 
decisions regarding the adaptation. The client simply reports certain 
measurements that the server interprets.

- Client controlled: The client makes the decisions and interpretations 
of what it see. Then it issues commands to the server that it must 
follow. The server is a simple slave.

- Hybrid control: In a hybrid control scenario both client and servers 
makes decisions and interpretations of the data.

I will not try to compare these solutions this if for future 
discussions. If one is interested in a server controlled solution one 
can look at the latest proposal in 3GPP SA4: 
http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_27/Docs/S4-030501.zip
Look at the buffer reporting mode variant. This proposal provides 
receiver buffer control and content rate control. The transmission rate 
control is handled through RTCP reports with enabled extended reports of 
packet loss vectors.

Some more direct comments on the draft:

1. Introduction: The encoding could be done in real time allowing 
maximum flexibility in the produced content rate. However this is may 
not result in optimal encodings and are also very resource demanding for 
a server.

2. Section 2.1.1 and 2.1.2: I think the behaviour in this case should 
really be depending on the operators policies. I would not like to have 
my expensively paid on demand video reduced in quality if this network 
is of the type you describe. However I can see that we should enable this.

3. Section 2.2: I think you need to be a bit more clear that this is one 
example of a service. Also the AMR rate of 5 kbps is a bit to low. The 
4.75 kbps mode has a quality that is not the best. However to achieve 
this bit-rate you must only send one AMR packet every 1.28 seconds, 
which means 64 amr frames per packet. This is not realistic as every 
packet will be required to reach the receiver to achieve acceptable 
quality. I would think that reasonable AMR qualities are using at least 
10 packet every second, meaning that a header overhead of 3 kbps.

4. Section 2.5: I think this section assumes something that are not 
quite true. First the impossibility to signal this rate switch using 
current protocols. I claim it is possible but most likely not supported 
by any available client. To signal the rate switch in SDP you present 
one different RTP payload types for each codec and configuration, most 
likely at least 4 maybe 5. The 40 and 50 kbps AAC stereo may be possible 
to encode using the same AAC configuration. The declared payload types 
are present in a SDP. A client shall not accept the session and setup 
the media stream unless it can handle all these codecs. If it has setup 
this media stream it has in fact accepted that any of the codec may be 
used in the media stream.

The server can here use different strategies when switching. The most 
likely one is to use a single SSRC and in the same RTP sequence number 
and timestamp space send the selected media quality. When the server 
switched this will be indicated by the new RTP PT. This has a slight 
problem as RTP/RTCP does not handle switched RTP TS rate perfectly. The 
client may in fact lose synchronization until it receives a RTCP packet 
after the switch has taken place. This happens if the packet where the 
RTP TS rate switches is lost. This is not a problem as long as the RTP 
TS rate before and after the payload type switch has the same TS rate. 
In other words not a problem for any video codec which all runs at 
90000. This is more a problem for audio codecs. I would in fact try to 
address this problem with audio codecs by ensuring that any audio codec 
is allowed to select a RTP TS rate that is a even rate multiple of its 
native value. This would allow audio codecs to run at a the smallest 
common denominator rate.

The other strategy at points of switching would be to use different 
SSRCs. This would violate the semantics that a SSRC represents a certain 
source. This also has some problems with the synchronization of the 
different SSRCs. This can't be done in RTSP as RTSP due to an erroneous 
definition of RTP-Info did not map each synchronization information to 
the SSRC. This can only be fixed through a new header due to backwards 
compatibility problems. Otherwise RTCP could be used. By sending SR for 
all SSRC one could in fact ensure that the synchronization is available. 
It will utilize more RTCP bandwidth.

5. Section 3.1.2: I don't think you should mix congestion control with 
adaptation control that hard. In my opinion, how drastic a content rate 
change is needed and how much depends on the clients buffer status.

6. Section 3.2.1: I think this directly falls under Transmission rate 
control and is a function of the congestion control mechanism.

7. Section 3.2.2.1: I don't think that this is a valid requirement. This 
is clearly an option that must be available. The controlling entity can 
trade off sending new data against retransmitting lost data. This trade 
off needs to be available to maximize quality. Further this can be done 
without causing congestion as long as the sender keeps to the available 
network rate. This is why splitting the functionality into different 
functionality blocks are so important. By having this model one can see 
that a server can have network rate N(t), content rate C(t) and 
retransmission rate R(t). The controlling entity must ensure that 
C(t)+R(t) <= N(t). Further the clients buffer allows that on short time 
scales R(t) can be equal to N(t) as long as enough data are available in 
the clients buffer.

8. Section 3.7: I don't think we can deploy a solution for switching 
that gives full functionality and at the same time is backwards 
compatible. However think on how one can maintain some of the 
functionality and being backwards compatible may be a worth the effort. 
However this is rather a design evaluation criteria than a requirement.

Best Regards

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: "Gamze Seckin" <[email-masked]>
Date: Wed, 16 Jul 2003 02:27:18 -0700

hi Philippe,
 
I agree with Magnus's comments on differentiating content rate from transmission rate and keeping an eye on reciever buffer overflow and underflow. These are all valid and useful discussions but I believe are out of the scope of the stream switching documents (please correct me if I am wrong).I believe the draft tries to cover far too many requirements that are rate control related (which have been/are discussed in other wg's). 
 
I personally would expect these two drafts on stream switching to be more specific on codec change support and addressing the issues of the transport (more serious problems that need an answer).
I think section 2.5 is the core of the document, since video stream switches are really not a big issue, whereas audio switches require more complicated solutions that may impact the user experience. And hence we should discuss and recommend solutions for those..
 
thanks
-gamze seckin

	-----Original Message----- 
	From: Magnus Westerlund [mailto:[email-masked]] 
	Sent: Mon 7/14/2003 5:40 AM 
	To: [email-masked] 
	Cc: [email-masked] 
	Subject: Re: [MMUSIC] stream switching

	Hi Phillipe,
	
	I hope that I can clarify anything that was unclear. I also has some
	further comments, see below.
	
	[email-masked] wrote:
	
	>Magnus, you wrote:
	>   yes
	>   yes, this is fundamental for seamless-ness
	>
	> 
	>
	>   I am not sure about this "must", could it be that your are
	>   considering differently media rate and transmission rate ?
	> 
	>
	
	What I mean is that if the available transmission rate is less the media
	rate then you on some time scale if nothing changes you will empty your
	buffer. Therefore you "must" change the media rate to avoid this or have
	interrupts. I don't know about any other solution.
	
	>   because I am not considering changing the transmission rate (as a
	>   end-to-end application issue) => stream switching is only about
	>   changing the *media* rate.
	>
	
	I don't think you can exclude the transmission rate from the equation.
	In my opinion you need to consider transmission rate, receiver buffers,
	and the media rate all together.
	
	>
	>   what I want to mean is you dont *need* to increase the media rate
	>   to avoid undeflows but instead you may *want* to increase the
	>   media rate in order to improve the user experience in terms of
	>   image quality...
	>
	I am not sure what you mean, but I guess the following. In cases you are
	not transmission rate limited one is not required to increase the media
	rate, however it is commonly desired. I can agree that there are some
	need to allow the receiver to specify what is allowed in respect to
	media rate modifications.
	
	>   hum, again i prefer the term "media rate control" if you dont mind?
	>
	
	No, I don't think you completely understand. In a system where the
	buffer space is known you don't have to transmit the data in the same
	rate as the media rate. Due to the buffer one can during some parts of
	the session transmit data at rates that are both higher and lower than
	the media rate. However when doing this you need to ensure that you are
	not under or overflowing the receiver buffer. What you try to do is to
	simplify this system to basically say that the transmission rate is
	equal to media rate which I think prevents better performance.
	
	If any one does not understand this, please prod me and I will try to
	provide a better explanation of the complete system.
	
	>   yes, not sure there is much to "do" there in terms of IETF spec?
	>
	
	There is nothing in IETF that allows feedback or even specification
	about the receiver buffers to or by the sending side. This is analogous
	to TCP's receiver window that allow to control the flow of media.
	
	>   yes! this is stream switching principle, but again it is about
	>   "media rate", not changing the transmission rate.
	>
	
	This I can agree is what you describe as media rate control. By
	selecting or controlling the media rate one can meet the requirements of
	transmission rate and receiver buffers.
	
	>>of the functionality's they address.
	>>
	>>   
	>>
	>
	>   Which groups would you suggest ?
	>
	
	Some suggestions would be:
	
	Media requirements: Requirements to prevent the media from being
	corrupted or worsen quality, like limited packet losses, gap free
	switching, RAP,etc.
	
	Signalling requirements: Requirements that are a related to what the
	signalling mechanism needs to achieve.
	
	System requirements: Requirements that have more general feeling, like
	backwards compatibility, etc.

	>   slave, yes, but a slave that *could* refuse to obey ;-) e.g. as
	>   discussed in my draft for rate control a server must be allowed to
	>   deny a request for a higher data rate...
	>
	> 
	>
	In a true client control paradigm, this may not even be true. But the
	basic point is that the requirements may motivate the choice, but we
	should not choice a a paradigm and then write the requirements to fulfil
	them. I have a rather long list of comments to when this discussion
	needs to be started.
	
	>   I have to say that using such RTCP extensions looks very much like
	>   "client control" to me ;-) but I dont really care about the
	>   terminology ;-)
	>
	
	Actually the mechanism in my opinion is very server controlled. The
	client provides the server side with some parameters telling it what to
	achieve. Then the client provides regular feedback to allow the server
	to run the algorithms and control the whole system.
	
	>   yes, I did not mention this possibility because in large scale
	>   media distribution this would effectively require one "production"
	>   encoder per client, which is -even in several years from now- way
	>   beyond the cost points for this type of products.
	> 
	>
	I think that spending two sentences on this is useful, just to make it
	clear that it has been considered but has significant problems.
	
	>   Yes, you are right, all this assumes the correct commercial
	>   agreements are in place. Our experience is however that a end user
	>   always prefers a continuous playback, even a degraded one,
	>   providing that the degradation is "resonable", all other solutions
	>   including "freezes" for rebuffereing are not acceptable for the
	>   wide public who is accustomed to TV ;-(
	>
	>From an IETF point of view (hey, just giving my own opinion ;-)
	>I think it would be a good idea that implementations *by default*
	>would "behave a la TFRC" rather than what we have today,
	>because otherwise -as sally floyd pointed out this morning-
	>we would run into rather nasty trouble.
	>
	
	Yes, I agree that over any best effort network you need some type of
	congestion avoidance. So default behaviour is good, however we must also
	allow for systems that has QoS agreements.
	
	>   right.
	>>current protocols.
	>   What *is* impossible is to find a comprehensive spec for stream
	>   switching !-) and it is my intention to provide one, i would love to
	>   "use the current protocols"!
	>
	> 
	>
	>>I claim it is possible but most likely not supported
	>>by any available client.
	>>   
	>>
	>
	>   And that would be the point in writing that spec...
	>
	> 
	>
	>>To signal the rate switch in SDP you present
	>   well, I completely agree that *that* may be a solution to stream
	>   switching, the point then would be to document it!-)
	> 
	>
	
	Good, then you should rather document this deficiency rather than
	claiming that it is possible.
	
	>>and timestamp space send the selected media quality.
	>>   
	>>
	>
	>   agreed, this is a very nice possibility...that obvioulsy we would
	>   need to *seriouly* document ... because this is not the "default"
	>   thing at all from existing specs, right?
	>
	> 
	>
	
	I have seen it as being the default behaviour, however this is nothing
	that has been thoroughly discussed in AVT and the RTP spec not be that
	clear on this point. Documenting this would be a good thing.
	
	>>When the server
	>>after the switch has taken place.
	>>   
	>>
	>
	>   right! that is a key issue!
	>
	> 
	>
	>>This happens if the packet where the
	>>90000.
	>   yes, not a problem *if* (as mentioned above) you mandate the same
	>   RTPTS space for all streams ...
	>
	> 
	>
	>>This is more a problem for audio codecs. I would in fact try to
	>   Is it always possible/efficient?
	>
	
	No, not for the currently specified RTP payload formats. It is something
	I will recommend in future payload formats to allow for this functionality.
	
	>Hum, so we should disregard this option?
	>
	
	I think so.
	
	>sorry, I do not understand that one?
	>
	
	This has to do with the complete system that a sender can transmit at
	one rate, while the media rate has another rate. The discrepancy is
	handled by the receiver buffer. Therefore the control of the media rate
	can be performed more drastic or not depending on what the buffer status
	is. An almost empty receiver buffer needs to be handled quickly and
	drastically, probably be reducing the media rate significantly. If the
	buffer are rather full the handling of this can be much less drastic, a
	small discrepancy can simply be tried to waited out. Or the quality
	change can be minimal.
	
	>   do you mean it is not appropriate to have backward compatibility as
	>   a requirement?
	>
	> 
	>
	
	It might, however I think it should be further considered. I think
	listing the level as critical is to high level.
	
	Cheers
	
	Magnus Westerlund

&��"rf��)��+-�k��Ȟ��+�m����5��_��?�������+-�w��i��'

---

From: [email-masked]
Date: Wed, 16 Jul 2003 12:37:38 +0200

Gamze,

Thank you for your feedback!

> I believe the draft tries to cover far too many requirements
> that are rate control related (which have been/are discussed in other wg's).

I am sorry it gives this impression.
rate control is EXPLICITELY out of the scope.
clearly rate control is dealt with using TFRC, DCCP etc

====The scope is purely media/codec control.====

I tried however to be exhaustive about the context
and of course rate control is at the hearth of it ;-)

>... since video stream switches are really not a big issue,

I am *extremely* surprised you can say this,
did you try to feed a MPEG-4 video stream into a H263 decoder recently ?
what about screen size changes?
when a video decoder tries to write into a display buffer
an image larger than the buffer the result is never pretty to see!-)

> whereas audio switches require more complicated solutions that may impact the user experience.

yes, audio is totally critical since it does not suffer any gap.

> And hence we should discuss and recommend solutions for those..

I would be horrified to limit the scope to audio!
the point on the contrary is to be able to switch streams of all types,
audio, video, text, you name it.
(streaming text is a use case that is missing in the document)

(I dont think anybody will ever need to switch text streams
for rate control reasons however) there is the case where the user wants to
switch from the italian subtitles into the german subtitles.

regards,

Philippe Gentric
Software Architect
Philips MP4Net
[email-masked]
http://www.platform4.philips.com

                                                                                                                                       
                                                   To:   "Magnus Westerlund" <[email-masked]>                          
                                                    Philippe Gentric/MP4-SUR/CE/PHILIPS@EMEA1                                          
                                                   cc:   <[email-masked]>                                                             
                                                   Subject:    RE: [MMUSIC] stream switching                                           
               "Gamze Seckin"                                                                                                          
               <[email-masked]>                Classification:                                                                     
                                                                                                                                       
               Sent by:                                                                                                                
               [email-masked]                                                                                                   
                                                                                                                                       
               16/07/2003 11:27                                                                                                        

hi Philippe,

I agree with Magnus's comments on differentiating content rate from transmission rate and keeping an eye on reciever buffer overflow and underflow. These are all valid and useful discussions but I believe are out of the scope of the stream switching
documents (please correct me if I am wrong).I believe the draft tries to cover far too many requirements that are rate control related (which have been/are discussed in other wg's).

I personally would expect these two drafts on stream switching to be more specific on codec change support and addressing the issues of the transport (more serious problems that need an answer).
I think section 2.5 is the core of the document, since video stream switches are really not a big issue, whereas audio switches require more complicated solutions that may impact the user experience. And hence we should discuss and recommend solutions for
those..

thanks
-gamze seckin

             -----Original Message-----

             Hi Phillipe,

             I think so.

             Cheers

             Magnus Westerlund