Subject: RE: [AVT] RE: <draft-ietf-avt-rtp-vmr-wb-03.txt>: sampling rate

From: [email-masked]
Date: Fri, 10 Sep 2004 12:17:50 -0700

Hi Qiaobing,

> >> 1- Only one RTP timestamp clock rate of 16000 Hz is used 
> throughout 
> >> the I-D.
> > 
> > I think this is a good change that simplify things.
> 
> Agreed. I'd also like to suggest that some new text be added 
> to capture the essence of this 
> sampling rate discussion... we don't want further readers of 
> the RFC to get the wrong 
> impression that only 16k speech/music can be passed through 
> vmr-wb codec. Some explanation 
> over why only 16k timestamp rate is defined for the payload 
> and how one can still work with 
> 8k media would definitely help, especially to those who are 
> unfamiliar with the codec 
> internal operation.
> 

Looks like that we have reached to an agreement. I will include some text in the draft to address your point, something like " although a 16000 Hz RTP clock rate is used for VMR-WB, the injection and processing of 8000 Hz narrowband media is also allowed and can be handled by VMR-WB decoder".

Regards

-Sassan Ahmadi

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Colin Perkins <[email-masked]>
Date: Sat, 11 Sep 2004 13:06:54 +0100

On 10 Sep 2004, at 20:17, [email-masked] wrote:
> Looks like that we have reached to an agreement. I will include some 
> text in the draft to address your point, something like " although a 
> 16000 Hz RTP clock rate is used for VMR-WB, the injection and 
> processing of 8000 Hz narrowband media is also allowed and can be 
> handled by VMR-WB decoder".

We have not reached agreement on this.

Colin

---

From: Qiaobing Xie <[email-masked]>
Date: Sun, 12 Sep 2004 02:07:33 +0800

Colin,

I think this is all because of VMR-WB's unique built-in "rate normalization" capability. 
This sort of makes the codec operation __rate independent__, as already talked about 
repeatedly in this discussion. Personally, I think this as an advancement in codec design. 
Among other advantages, this allows some simplification in session setup signaling, 
additional flexibility in sys architecture, and enhanced interoperability, and people in 
3GPP2 want to take advantage of this.

Now the question is how to effectively support this in the payload (I think the current 
draft has crippled this capability).

regards,
-Qiaobing

Colin Perkins wrote:

>>>>> 1- Only one RTP timestamp clock rate of 16000 Hz is used throughout
>>>>> the I-D.
>>>>

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 13 Sep 2004 16:41:58 +0200

Hi Sassan and Colin,

I think we have two issues:

A. Is there any benefit to indicate or request that the sampling 
frequency used at the sender.

B. Is it necessary to use the sampling frequency as RTP timestamp rate.

I will start with A that I think is easier to explain and also can 
provide some information for issue B. If you find any of my assumptions 
and statements are incorrect, please correct me.

To my understanding of the VMR-WB after a conversation with Jonas 
Svedberg is that the VMR-WB will provide a somewhat better encoding of 
8kHz material if it is indicated that the input is 8kHz. However there 
is no need due to compatibility or decoder operation to signal the case 
where the 8kHz is used as input into the encoder. These would then 
result that the only case needed to be signaled between encoder and 
decoder is cases where the decoder will use output at 8kHz. Because if 
the decoder can request that the encoder uses 8kHz input some 
improvement of the 8kHz material is achieved. In the other cases where 
the receiver is capable of 16kHz it doesn't matter for the receiver if 
the original audio was 8 or 16kHz from a decoding point of view.

Colin, if one looks at issue B. Is it really needed to use the RTP 
timestamp frequency equal to the sampling rate used? I would say NO to 
that question. My reasoning is the following.

- Many audio input is sampled from a source at a higher rate then the 
encoder may handle. Thus a resampling and pre-processing stage is 
employed based on the encoders input frequency rather then producing 
that rate initially from the hardware. Some of the reason is that the 
pre-processing may actually yield better results than what the hardware 
at given input rate can gain. Another reason may be that one like to 
avoid switching the hardware between rate if changing the encoding.

- The frame based decoders does not need to know the encoders input 
rate. The encoder may anyway resample this into other rates for internal 
processing and band limited signals. I would claim that VMR-WB, AMR-WB+ 
and AAC are all example of codecs that perform this kind of tricks. On 
the receiver side they produce a output signal that has any sampling 
frequency the receiver finds most useful. Either causing clipping of the 
higher frequencies, but more commonly to a higher clock rate, despite 
that no more information is provided simply for ease of use.

- The frame based codecs do only need a RTP timestamp that allows the 
receiver to correctly reconstruct the time line when the encoding is 
done with the most audio bandwidth. In the VMR-WB case this is 16kHz. 
AMR-WB+ is even more strange, as we have selected an RTP timestamp rate 
that results in that all internal sampling frequencies will result in 
integer timestamp ticks. Thus actually allowing one to correctly 
calculate frame alignment when the internal sampling frequency changes. 
That the frequency also is possible to recalculate into several common 
sampling frequencies with few partial sample alignments was also 
considered.

Thus I would use this to argue that indicating the actual sampling 
frequency is not necessarily as long as the receiver is capable of 
correctly reconstruct the media stream with its timing information in 
full resolution.

In the VMR-WB case I would think that having only one timestamp rate of 
16kHz does not effect codec operation and would simplify the handling 
when one has some senders that do use 8kHz, especially when gateways 
need to encoded sometime 8kHz material from pre-recorded responses and 
in other cases WB channel data. This do avoid the need to perform RTP 
timestamp rate switches.

If desired to have this possibility to request by a receiver that the 
sender do use 8kHz input then one should introduce a MIME parameter for 
this. However I would like to avoid using the "rate" parameter as it 
results in unnecessary barriers in form of signalling and RTP timestamp 
rate switching.

Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: "Alan Clark" <[email-masked]>
Date: Mon, 13 Sep 2004 12:57:01 -0400

We've started to build a database of speech samples in multiple languages
using (roughly) phonetically balanced source material.  The site currently
contains around 70 audio files in English (American and British), French,
Mandarin and Hindi, and we are in the process of adding the source text for
each language to the site (text editors allowing!!!).  For English we've
used Harvard Sentences, France Telecom provided some French test sentences
and we've constructed some new Hindi and Chinese sentences.

The goal of the site (www.voiptroubleshooter.com/open_speech/) is to provide
freely usable, non-copyrighted material that can be used for VoIP testing
and research, published on web sites etc., as much of the previously
available speech material has some restriction on use. We would welcome
comments, offers to provide new material etc.

Regards

Alan Clark
Telchemy

---

From: Colin Perkins <[email-masked]>
Date: Thu, 23 Sep 2004 09:58:48 +0200

Hi,

> frequency used at the sender.

Yes. This is why RTP has the "rate" parameter, and uses the sampling 
rate as the RTP timestamp rate.

> B. Is it necessary to use the sampling frequency as RTP timestamp rate.

It's highly desirable.

> provide some information for issue B. If you find any of my 
> assumptions and statements are incorrect, please correct me.
>
> is no need due to compatibility or decoder operation to signal the 
> case where the 8kHz is used as input into the encoder. These would 
> then result that the only case needed to be signaled between encoder 
> and decoder is cases where the decoder will use output at 8kHz. 
> Because if the decoder can request that the encoder uses 8kHz input 
> some improvement of the 8kHz material is achieved. In the other cases 
> where the receiver is capable of 16kHz it doesn't matter for the 
> receiver if the original audio was 8 or 16kHz from a decoding point of 
> view.
>
> that question.

Yes, it is necessary to use an RTP timestamp equal to the sampling rate.

> My reasoning is the following.
>
> pre-processing may actually yield better results than what the 
> hardware at given input rate can gain. Another reason may be that one 
> like to avoid switching the hardware between rate if changing the 
> encoding.
>
> rate. The encoder may anyway resample this into other rates for 
> internal processing and band limited signals. I would claim that 
> VMR-WB, AMR-WB+ and AAC are all example of codecs that perform this 
> kind of tricks. On the receiver side they produce a output signal that 
> has any sampling frequency the receiver finds most useful. Either 
> causing clipping of the higher frequencies, but more commonly to a 
> higher clock rate, despite that no more information is provided simply 
> for ease of use.
>
> AMR-WB+ is even more strange, as we have selected an RTP timestamp 
> rate that results in that all internal sampling frequencies will 
> result in integer timestamp ticks. Thus actually allowing one to 
> correctly calculate frame alignment when the internal sampling 
> frequency changes. That the frequency also is possible to recalculate 
> into several common sampling frequencies with few partial sample 
> alignments was also considered.
>

True, but it greatly simplifies the system if all codecs use the 
sampling rate as the RTP clock rate. You can make things work if each 
codec uses a different rate, but it's desirable that RTP is consistent 
where possible. Why is this codec so special that it needs to break 
this rule?

> In the VMR-WB case I would think that having only one timestamp rate 
> of 16kHz does not effect codec operation and would simplify the 
> handling when one has some senders that do use 8kHz, especially when 
> gateways need to encoded sometime 8kHz material from pre-recorded 
> responses and in other cases WB channel data. This do avoid the need 
> to perform RTP timestamp rate switches.

But in the process you make senders that support multiple codecs more 
complex, since they can't use the sampling rate to drive the media 
clock for all codecs.

> sender do use 8kHz input then one should introduce a MIME parameter 
> for this. However I would like to avoid using the "rate" parameter as 
> it results in unnecessary barriers in form of signalling and RTP 
> timestamp rate switching.

I disagree. Using the rate parameter is consistent with other payload 
formats, and so will simplify the system overall.

Colin

-- 
Colin Perkins
http://csperkins.org/

---

From: Qiaobing Xie <[email-masked]>
Date: Fri, 24 Sep 2004 19:53:14 +0800

Hi, Colin,

Colin Perkins wrote:

<snip>
> complex, since they can't use the sampling rate to drive the media clock 
> for all codecs.

I'd think that the saving (in terms of reduction to the system design and signaling 
complexity) by avoiding performing RTP timestamp rate switches (as Magnus pointed out above) 
far overweighs the ability of using the sampling rate to drive the media clock.

regards,
-Qiaobing

---

From: Magnus Westerlund <[email-masked]>
Date: Fri, 24 Sep 2004 14:00:28 +0200

Qiaobing Xie wrote:
> 
> I'd think that the saving (in terms of reduction to the system design 
> and signaling complexity) by avoiding performing RTP timestamp rate 
> switches (as Magnus pointed out above) far overweighs the ability of 
> using the sampling rate to drive the media clock.
> 

Also, nothing in the case of VMR-WB prevents it from driving the media 
clock. The larger issue is that VMR-WB has built in resampling 
functions, and the fully information carrying sampling rate needed is 
16kHz, not 8. However resampling before the codec, or resampling in the 
codec does not make any difference from the handling, only somewhat in 
regards to quality, due to encoder optimization.

As the native out-put is 16kHz using a 8kHz RTP clock prevents a 
receiver from driving its media clock with the appropriate rate. If a 
receiver is using 8kHz output, then it easier to scale down the rate, 
then to scale it up for the normal receiver behavior.

Thus I see the least problem of driving the media clock when we only 
uses a 16kHz clock, rather then a clock that reflects the encoders 
front-end.

Cheers

Magnus Westerlund

---

From: Randell Jesup <[email-masked]>
Date: 24 Sep 2004 15:15:02 -0400

Colin Perkins <[email-masked]> writes:
>Yes. This is why RTP has the "rate" parameter, and uses the sampling rate
>as the RTP timestamp rate.

        My apologies, but I don't see how that answers the question as to
what the benefit is.  How does the receiver make use of this information?
Why is this better for the receiver?

        I can see some advantage in having the RTP rate parameter be the
normal _output_ sample rate from the codec (which may be uncorrelated to
the input rate!) in that you could use that to implement an audio codec
with non-fixed frame sizes.

>> B. Is it necessary to use the sampling frequency as RTP timestamp rate.

        Again, asserting this without a reason doesn't convince me.

>Yes, it is necessary to use an RTP timestamp equal to the sampling rate.

        Again, no reasons given.

>> employed based on the encoders input frequency rather then producing that
>> rate initially from the hardware. Some of the reason is that the
>> receiver to correctly reconstruct the time line when the encoding is done
>> with the most audio bandwidth. In the VMR-WB case this is 16kHz. AMR-WB+
>> is even more strange, as we have selected an RTP timestamp rate that
>> results in that all internal sampling frequencies will result in integer
>> timestamp ticks. Thus actually allowing one to correctly calculate frame
>> alignment when the internal sampling frequency changes. That the
>> frequency also is possible to recalculate into several common sampling
>> frequencies with few partial sample alignments was also considered.
>>
>True, but it greatly simplifies the system if all codecs use the sampling
>rate as the RTP clock rate. You can make things work if each codec uses a
>different rate, but it's desirable that RTP is consistent where
>possible. Why is this codec so special that it needs to break this rule?

        Again, I simply don't see how things are simplified by having the
RTP timestamp rate be the sender-side sample rate.  If the timestamp rate
is 2x or 4x the input rate, or if it's the lowest common multiple (if you
know what I mean) of a range of sample rates the encoder might use, how
does that hurt the receiver?  In fact, if it _is_ possible for the encoder
to use multiple rates (like VMR-WB), isn't it a lot easier on both sides
if we can just us 16KHz for RTP and avoid having the play games if we want
to switch input rates dynamically?  Setting the timestamp at the sender end
is trivial.

        Also, sticking with a strict timestamp rate == sample rate dictate 
means that various classes of codecs won't fit well, will be harder to
write and/or use with RTP, or won't be developed because the transports
for them would be painful.  Or, perhaps worse for the participant of this
group, such codecs will push the use of alternative mechanisms such as
IAX2 or other possible alternatives.  I could simplify my life CONSIDERABLY
by ignoring RTP and using a single multiplexed stream, with lower overhead
due to piggybacked nacks and acks, and much less hassle blowing multiple
holes through firewall/NATS.  But for the sake of standards and
compatibility I've gone through the pain of getting dealing with all the
issues surrounding RTP, feedback, RTP-through-NATs, etc, and have probably
hewn (much) closer to the specs than most, I suspect.

        Don't put up roadblocks where there isn't a reason to.  In this
case, there is a good argument _for this codec_ to use a fixed 16K
timestamp rate.

>> when one has some senders that do use 8kHz, especially when gateways need
>> to encoded sometime 8kHz material from pre-recorded responses and in
>> other cases WB channel data. This do avoid the need to perform RTP
>> timestamp rate switches.
>

The sender "can't use the sampling rate to drive the media clock for all
codecs"?

        Worst case (since this may fall out of all the tons of other work
the sender is doing) is the sender does:

        rtp_packet->timestamp = first_sample_time * conversion_for_codec;

(and no it doesn't have to use floating point even if it's fractional, of
course).

        More typically, I suspect for frame-oriented codecs the code will
initialize the RTP stack with the 'normal' timestamp increment, or
calculate that value once and store it in a variable to pass with each
packet to the RTP code.  Not to mention Magnus's comments about senders
who actually sample at much higher rates and downsample for encoding.

        My apologies for disagreeing, but I don't see any serious attempt
to answer either Magnus's, other commenter's or my arguments and examples
on this issue; I merely see assertion that "that's the way it's done", and
"it's simpler" without significant explanation (which we've disagreed with
and given arguments to).

-- 
Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team
[email-masked]