Subject: [SIP] Format changes but doesn't call pre-access codec's setInputFormat() method!

From: "Victor A Paulsamy" <[email-masked]>
Date: Mon, 22 Jan 2001 16:04:21 -0500

Hi Ivan,

PFA a mail, in which you wrote "Right when the media engine detects the
format change, your pre-access codec will be notifed with the
Codec.setInputFormat() call with the new format," however, that is not the
case with my application.

I'm working in an application which detects Comfort Noise (CN) and handles
it. As per my understanding, non-voiced activity falls under CN thus a
"RemotePayloadChangeEvent" should signify switching formats between audio
format and CN.  However, I'm faced with the following two situations when
the "RemotePayloadChangeEvent" occurs and the format changes from audio
format.

Situation I. Pre-access codec is not called.
Situation II. Pre-access codec is called; the new format is CN.

The application receives/makes phone calls thro' Gateway. "Situation I"
typically happens in the pauses in phone rings and also when a user pauses
between voice activity.

In other words the number of occurrences of "RemotePayloadChangeEvent" is
*not equal* to the number of calls made to pre-access codec (the former
being more frequent).

Please help me understand what might happen in "Situation I"?

Warm regards,

--victor

-----Original Message-----
From: Discussion list for Java Media Framework API
[mailto:[email-masked]]On Behalf Of Ivan Wong
Sent: Thursday, February 01, 2001 8:44 PM
To: [email-masked]
Subject: Re: [JMF] New Audio Format / Comfort Noise / Custom Codec

Here some info for the record:

After exchanging a few emails with Anshul, I suggested removing
DirectAudioRenderer and use JavaSoundRenderer instead.  He
experimented with that and got better results.  Here's how to
switch off DirectAudioRenderer:

1) Run JMFRegistry;
2) Select the "PlugIns" tab;
3) Select the "Renderer" tab;
4) On the scrolling list, select
   com.sun.media.renderer.audio.DirectAudioRenderer;
5) Click "Remove"
6) Click "Commit"

JavaSound uses a large mixing buffer and is better at handling
data loss which is exactly the problem since the added codec
artificially injects breakup to the audio stream.  However,
the price for that is long latency.

We'll work on the DirectAudioRenderer to make it better
at handling data loss.  But there may still be a tradeoff
between latency and quality...

Ivan Wong
Java Media Engineering
Sun Microsystems, Inc.

> MIME-Version: 1.0
> Content-Transfer-Encoding: 7bit
> X-Priority: 3 (Normal)
> X-MSMail-Priority: Normal
> Importance: Normal
> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
> Date: Thu, 1 Feb 2001 16:46:08 -0500
> From: Anshul Choudhry <[email-masked]>
> Subject: Re: [JMF] New Audio Format / Comfort Noise / Custom Codec
> Comments: To: Ivan Wong <[email-masked]>, [email-masked]
> To: [email-masked]
>
> Thanks for the reply Ivan.
>
> I did get the thing working as you suggested but there are still some open
> issues.
>
> 1) I have to call  DataSource.start() every time the payload changes.
> 2) The setInputFormat is called only once for both for Format19 and format
> 4. But I do get a payload change event in update( ReceiveStreamEvent
event).
> 3) I do "fool" the media engine and it works.
>
> Still the following problems remain.
>
> 1) The reception is OK as long as format does not change. As soon as it
> changes I get a "broken echo" of last one second of reception. And
sometimes
> I get "broken echo" of the "broken echo". I have been trying all sort of
> tricks to do away with it. But as yet I do not understand why it comes !!
>
> 2) Since the format is changing rapidly. The quality of voice is not too
> good. A little amount of speech is cut every time the format changes. I
> think it is the delay in the starting of datasource.
>
>
> Ironically once I had no comfort noise coming from the transmitting side.
I
> had wonderful reception.
>
> I will appreciate your help on any of the questions above.
>
> regards,
>
> Anshul
>
>
>
>
> -----Original Message-----
> From: Ivan Wong [mailto:[email-masked]]
> Sent: Tuesday, January 30, 2001 9:50 PM
> To: [email-masked]; [email-masked]
> Subject: Re: [JMF] New Audio Format / Comfort Noise / Custom Codec
>
>
> Anshul,
>
> There's a couple of missing details:
>
> 1) Right when the media engine detects the format change,
>    your pre-access codec will be notifed with the
>    Codec.setInputFormat() call with the new format.  Your
>    codec needs to return the same format that's being set.
>    This is a way for the codec to reject the new format if
>    it's not supported.
>
> 2) In the Codec.process() call of your pre-access codec, you
>    need to explicitly set the format of the output buffer to
>    whatever the old format is.  This is to "fool" the media
>    engine that no format change had occurred since your pre-access
>    codec is shielding the rest of the data pipeline from the
>    new format change.  Otherwise, the new format change will
>    trickle down the data pipeline.  That will cause what you
>    saw there.
>
> There may be other problems.  But these are my guesses.
>
> Ivan Wong
> Java Media Engineering
> Sun Microsystems, Inc.
>
> > MIME-Version: 1.0
> > Content-Transfer-Encoding: 7bit
> > X-Priority: 3 (Normal)
> > X-MSMail-Priority: Normal
> > Importance: Normal
> > X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
> > Date: Mon, 29 Jan 2001 16:47:37 -0500
> > From: Anshul Choudhry <[email-masked]>
> > Subject: [JMF] New Audio Format / Comfort Noise / Custom Codec
> > To: [email-masked]
> >
> > - I created a new AudioFormat for payload type 19.
> > - Registered it with RTPManager.
> > - Modified the FrameAccess example to take care of the new AudioFormat.
> >
> > The idea was to track & filter the CN data with G711/G723 data (by
> > setDiscard(True) for all frames of CN data).
> >
> > The logic did not work however. Following is the log.
> >
> > PAYLOAD CHANGED..19
> > Track 0 is cn, 8000.0 Hz, 8-bit, Mono
> > .....................................
> > Failed to build a graph to render the input format with the given custom
> > options
> > .
> > Failed to realize: com.sun.media.ProcessEngine@41552289
> >   Cannot build a flow graph with the customized options:
> >     Unable to add customed codecs:
> >       com.VoiceCore.Prototype.FrameAccess$PreAccessCodec@245d2284
> >       com.VoiceCore.Prototype.FrameAccess$PostAccessCodec@25892284
> > Error: Unable to realize com.sun.media.ProcessEngine@41552289
> > PAYLOAD CHANGED..4
> > PAYLOAD CHANGED..19
> >
> >
> > What could possibly be wrong ? I appreciate your help.
> >
> > regards,
> >
> > Anshul
> >
> > PS : The goal remains the same. I want to play the incoming stream which
> is
> > alternating between format 4(G723) and format 19(Comfort Noise) say 5
> times
> > a second.
> >
> >
> >
> >
> > -----Original Message-----
> > From: Ivan Wong [mailto:[email-masked]]
> > Sent: Tuesday, January 09, 2001 1:17 PM
> > To: [email-masked]; [email-masked]
> > Subject: RE: [JMF] RTP Payload type 19
> >
> >
> > Anshul,
> >
> > Good question!  Mid-stream payload change are normally handled
> > by creating a new Player to handle the new format.  This is
> > acceptable since with the new format, the data processing pipeline
> > has to be rebuilt anyway.
> >
> > But comfort noise is such a special case that it should be handled
> > more gracefully within the framework.  We'll consider this case
> > more thoroughly on the engineering implications.
> >
> > As for now, I can think of one workaround.  But it's not that
> > trivial.  Here are the steps if you really want to try it:
> >
> > 1) Register a new format for Payload 19 with RTPManager.addFormat().
> >
> > 2) Write a custom preprocessing effect.  You can follow the
> >    FrameAccess sample on the solutions page:
> >
> >
> http://java.sun.com/products/java-media/jmf/2.1/solutions/FrameAccess.html
> >
> > 3) Doing something similar to PreAccessCodec in the sample, except
> >    in this case, it's use to shield the rest of the flow graph with
> >    the intermitent "comfort noise" data.  The idea is, when the
> >    input buffer's format changes to the new format representing
> >    payload 19, your codec can track that and do the necessary steps
> >    to handle the comfort noise data - by generating silence data; or
> >    simply ignore that data.
> >
> > Hope this helps somewhat.
> >
> > Ivan Wong
> > Java Media Engineering
> > Sun Microsystems, Inc.
> >
> > > -----Original Message-----
> > > From: Discussion list for Java Media Framework API
> > > [mailto:[email-masked]]On Behalf Of Anshul Choudhry
> > > Sent: Tuesday, January 09, 2001 7:10 AM
> > > To: [email-masked]
> > > Subject: [JMF] RTP Payload type 19
> > >
> > >
> > > Hello,
> > >
> > > How does JMF plan to handle payload type 19 (Comfort Noise) ?
> > >
> > > If I am receving audio in U-Law (Payload type 0) or G.723 and
> > > suddenly I get
> > > payload change event to payload type 19.
> > >
> > > As described in JMF I should remove the old player and create a
> > > new player.
> > > Add new target etc.
> > >
> > > * But the payload 19 lasts only for one second and rapidly
> > > changes between 0
> > > and 19.
> > >
> > > If I stick on to the old player. The audio can be heard no more.
> > >
> > > Even if JMF does not provide handling of payload type 19, could
somebody
> > > sugget a workaround ?
> > >
> > > Thanks,
> > >
> > > Anshul
> > >
> > > ==================================================================
> > > =========
> > > FAQ:
> > http://java.sun.com/products/java-media/jmf/forDevelopers/jmffaq.html
> > List Archive: http://archives.java.sun.com/archives/jmf-interest.html
> >
> > To unsubscribe, send email to [email-masked] and include in the
> body
> > of the message "signoff JMF-INTEREST".
> >
> >
>
===========================================================================
> > FAQ:
> http://java.sun.com/products/java-media/jmf/forDevelopers/jmffaq.html
> > List Archive: http://archives.java.sun.com/archives/jmf-interest.html
> >
> > To unsubscribe, send email to [email-masked] and include in the
> body
> > of the message "signoff JMF-INTEREST".
>
>
===========================================================================
> FAQ:
http://java.sun.com/products/java-media/jmf/forDevelopers/jmffaq.html
> List Archive: http://archives.java.sun.com/archives/jmf-interest.html
>
> To unsubscribe, send email to [email-masked] and include in the
body
> of the message "signoff JMF-INTEREST".

===========================================================================
FAQ:  http://java.sun.com/products/java-media/jmf/forDevelopers/jmffaq.html
List Archive: http://archives.java.sun.com/archives/jmf-interest.html

To unsubscribe, send email to [email-masked] and include in the body
of the message "signoff JMF-INTEREST".

_______________________________________________
This list is for continuing development of the SIP protocol.
The sip-implementor's list is the place to discuss implementation,
and to receive advice on understanding existing sip.
To subscribe to it, send mail to 
[email-masked] with "subscribe" in the body.

---

From: "Victor A Paulsamy" <[email-masked]>
Date: Mon, 22 Jan 2001 16:43:47 -0500

Hi Every one,

I apologize for the wrong posting!

Warm regards,

--victor

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Victor A Paulsamy
Sent: Monday, January 22, 2001 4:04 PM
To: Ivan Wong
Cc: SIP; Victor A Paulsamy
Subject: [SIP] Format changes but doesn't call pre-access codec's
setInputFormat() method!

Hi Ivan,

Warm regards,

--victor

-----Original Message-----

1) Run JMFRegistry;

Ivan Wong
Java Media Engineering
Sun Microsystems, Inc.

event).
sometimes
I
somebody
body