Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?)

From: "Pierre Forgues" <[email-masked]>
Date: Fri, 8 Jul 2005 11:36:13 -0400

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of David R Oran
Sent: Friday, July 08, 2005 10:05 AM
To: Dave Burke
Cc: [email-masked]; Shanmugham, Saravanan; Eric Burger
Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?)

On Jul 8, 2005, at 6:59 AM, Dave Burke wrote:

> I agree with your opinion on what constitutes a barge-in. Based on  
> Klaus' e-mail, I am more concerned that using the grammar type to  
> decide what constitutes a barge-in is going to make VoiceXML  
> implementations difficult.
>
> It is easy to map VoiceXML application selected inputmodes to MRCP  
> resources:
> a. inputmodes="dtmf" -> Use a dtmfrecog
> b. inputmodes="speech" -> Use a "speech-only-recog"
> c. inputmodes="both" -> Use a speechrecog (or a combination of  
> "speech-only-recog" + dtmfrecog)
>
> The only problem is (b). While not as important as being able to do  
> DTMF-only recognition, I believe we DO need to support speech-only  
> recongition so as (a) to avoid unnecessary limitations in VUI  
> design, and (b) to facilitate VoiceXML implementations. I think we  
> should add a header to RECOGNIZE so the client is able to always  
> _explicitly_ set the inputmodes.
>
I'm not sure I buy this, since what the recognizer is looking for and  
what constitutes an input that should be considered a potential barge- 
in strike me as independent. I'm similarly not persuaded that we need  
the flexibility to set the input mode independently of the grammar,  
since it leads to all sorts of inconsistent states (e.g. speech-only  
grammar with an input-mode of dtmf). On the other hand I can see the  
need for the client to specify what sorts of input out to generate  
the start-of-input (nee start-of-speech) event on.

Pmf> The mechanism for a client to specify the type of input is using
grammars.  These can have DTMF and/or speech requirements.  If you add
the complexity of an independent header to specify the input mode then
you will need to document the behavior for inconsistencies.

> In retrospect, I don't like the idea of multiple START-OF-SPEECH  
> events being generated from the same media resource. This is  
> because one assumes that the START-OF-SPEECH should be of the same  
> type as the hypothesis returned in the RECOGNITION-COMPETE message  
> - most implementations, on hearing one input mode type disable the  
> recogniser of the other type.
I'm not sure I follow this logic, but I'm not wedded to the idea of  
allowing multiple events. It was trying to solve the problem of  
ambiguity around what the recognizer was hearing and the idea that  
the client might care. If you don't think the client will ever care,  
then we don't need the capability.
Pmf> I agree we should not have multiple SOS events.  Too much chatter.

> I do like David's idea of renaming START-OF-SPEECH to something  
> like START-OF-INPUT and carrying a type header because it is neater  
> and more extensible.
>
> So in summary, I propose we modify the spec to:
>
> 1. Clarify what constitutes a barge-in for a dtmfrecog and  
> speechrecog as per Sarvi's e-mail (and in agreement with Klaus' for  
> dtmfrecog).
>
Hmmm, ok, but I think the issue is actually clarifying what the  
recognizer declares as "interesting input", which it may decide also  
constitutes a barge-in in the optimized case, and in either case  
reports to the client that it heard.
Pmf> I'm not going to argue changing the name of the event, but my
preference would be to keep existing method names unless there is a
clear reason for changing - which I do not see here.

> 2. Specifiy an InputModes header to RECOGNIZE (defaults to "both",  
> can also be "speech" or "DTMF"). Setting to speech for a  
> speechrecog results in the hypothesised "speech-only-recog".  
> Setting to DTMF for a speechrecog is equivalent to using a  
> dtmfrecog. Edge cases: Setting to speech for a dtmfrecog will  
> result in a noinput as will setting to DTMF for a speechrecog which  
> does not support DTMF.
>
I'm ok with having a header for the client to tell the server when it  
would like the start-of-input event to be generated and what the  
client considers to be the "interesting input" that the recognizer  
should use to do the discrimination, and possibly do barge-in  
processing in the optimized case.

I'm less ok with the proposed domain of values, since it will have  
extensibility problems. Especially problematical is have a code point  
of "both" since that will be ambiguous if we even define a "music"  
recognizer, or a "gesture" recognizer using video input. Here's my  
counter-proposal:

Create a header on the recognizer resource requests called "start- 
input-on:". Define a registry of values, with the following three  
values initially defined:
     - dtmf
     - speech
     - music

The semantics would be that the resource is to generate a "start-of- 
input" event, apply the defined grammars to what follows, and do any  
local optimized barge-in processing if ANY of the enumerated input  
types is detected. That way you can say
     "start-input-on: dtmf" if you want to just  dtmf,
     "start-input-on: speech" if you want just speech (dtmf input  
would be ignored even if the grammar supported dtmf)
     "start-input- on: speech, dtmf" if you wanted both
etc.

Pmf> Doesn't this come back to the previous point of having two
independent ways to specify input modes?  Through grammars and this
proposed new header?  If you add this then you will have to document
behavior on inconsistencies.  Anyway, if you proceed then I would
definitely make this optional

> 3. Change START-OF-SPEECH to START-OF-INPUT. Clarify it occurs once  
> and coincides with barge-in.
>
Ok for the "only once", but I'd like to tighten it up to talk about  
more than just barge-in, as I suggested above.

> 4. Add a header of InputType to START-OF-INPUT. Current specified  
> values are "dtmf" or "speech".
>
Ok, with slight modification. Say that the syntax of the "input-type"  
header is a single-valued subset of the registered value(s) that are  
defined for the start-input-on: header.

Comments?

Dave O. (technical hat on, chair hat off).

> Dave
>
>
>
> ----- Original Message ----- From: "Shanmugham, Saravanan"  
> <[email-masked]>
> To: "Eric Burger" <[email-masked]>; <[email-masked]>
> Sent: Thursday, July 07, 2005 5:52 PM
> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary 
> (?)
>
>
> I don't think generating multiple START-OF-SPEECH events is a  
> solution.
> We still haven't addressed, what constitutes a barge-in, for the
> optimised case. That should also be the single point when a single
> START-OF-SPEECH(or whathever else you want to name it) should be
> generated.
>
> That point, in my opinion should be
>   1. For "dtmf-recog" resources should be the beginning of a DTMF key
> press.
>   2. For "speech-recog" resources should be the beginning of a DTMF  
> key
> press or the beginning of speech. This is should be irrespective of  
> what
> type of grammar is being used. Coz even numbers only grammars can  
> still
> be spoken and hence cannot be assumed to be a cue for DTMF only
> recognition.
>   3. For "speech-only-recog" resources(which are not defined today)  
> the
> time to barge-in is the beginning of speech. I don't see a need for  
> such
> a resource today. But I am mentioning this for completeness.
>
> Sarvi
>
>
>     -----Original Message-----
>     From: [email-masked]
>     [mailto:[email-masked]] On Behalf Of Eric Burger
>     Sent: Thursday, July 07, 2005 5:32 AM
>     To: [email-masked]
>     Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode
>     -> summary(?)
>
>     Can we declare consensus?
>
>     > -----Original Message-----
>     > From: [email-masked]
>     [mailto:[email-masked]]
>     > Sent: Wednesday, July 06, 2005 10:45 AM
>     > To: Dave Burke
>     > Cc: [email-masked]; Shanmugham, Saravanan; Klaus Reifenrath
>     > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode ->
>     > summary(?)
>     >
>     > I think we're getting close. I though about snipping out
>     some pieces
>     > to cut down the text, but I realized the context is
>     still needed. See
>     > inline.
>     >
>     > On Jul 6, 2005, at 10:29 AM, Dave Burke wrote:
>     >
>     > > Inline.
>     > >
>     > > Dave
>     > >
>     > > ----- Original Message ----- From: "David R Oran"
>     <[email-masked]>
>     > > To: "Dave Burke" <[email-masked]>
>     > > Cc: <[email-masked]>; "Shanmugham, Saravanan"
>     <[email-masked]>;
>     > > "Klaus Reifenrath" <[email-masked]>
>     > > Sent: Wednesday, July 06, 2005 1:07 PM
>     > > Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only
>     mode -> summary
>     > > (?)
>     > >
>     > >
>     > >
>     > >>
>     > >> On Jul 6, 2005, at 5:31 AM, Dave Burke wrote:
>     > >>
>     > >>
>     > >>> + Attempting to summarise:
>     > >>>
>     > >>> 1. START-OF-SPEECH is useful for the client to know when
>     > to stop
>     > >>> playing prompts the in non-optimised case 2.
>     START-OF-SPEECH is
>     > >>> useful for the client to calculate the bargin  time
>     (e.g. VoiceXML
>     > >>> 2.1 <mark>)
>     > >>> 3. In the optimised case, a bargin automatically
>     stops prompt
>     > >>> playing (assuming prompts barginable) 4. Because of
>     the previous
>     > >>> point, the question of what input type  caused
>     bargin is different
>     > >>> and less important to
>     > what input
>     > >>> type(s)  the recogniser is listening for
>     > >>>
>     > >> I'm not sure I follow point 4. Could you elaborate?
>     > >>
>     > >
>     > > DB> Adding a parameter to the START-OF-SPEECH event would
>     > certainly
>     > > allow the client to ignore (i.e. let prompts continue
>     playing) the
>     > > event if the event type is not of interest (e.g. the
>     client would
>     > > ignore speech start events when it is interested only
>     in  DTMF start
>     > > events). This _only_ works for the non-optimised case,
>     however. For
>     > > the optimised case, assuming START-OF-SPEECH
>     > coincides
>     > > with the bargin signal to the speechsynth, prompts
>     will stop playing
>     > > for inputs that the client might not be interested
>     > in (e.g.
>     > > a speech input will stop prompts playing even if the client
>     > is only
>     > > interested in DTMF).
>     > >
>     > OK, now I get it. There's a need for the client to both
>     handle the
>     > non-optimized case itself, and influence or at least
>     have a clue what
>     > the server is going to do in the optimized case.
>     >
>     > >
>     > >>
>     > >>
>     > >>> 5. Currently, MRCPv2 has no way of indicating what
>     input type(s) a
>     > >>> recogniser is listening for
>     > >>>
>     > >>>
>     > >> Do you mean exactly this, or do you mean "for the client to
>     > >> indicate  to the resource what input types it should
>     look for"?
>     > >>
>     > >
>     > > DB> Yes exactly - apologies for not being clear.
>     > >
>     > >
>     > >>
>     > >>
>     > >>> + Why implement 5?
>     > >>>
>     > >>> i. Noisy case: Need DTMF-only recognition (and may
>     only have a
>     > >>> speechrecog)
>     > >>>
>     > >> I'm having difficulty following the logic of why
>     noise would
>     > >> necessarily trigger START-OF-SPEECH if you were listening for
>     > >> speech  but not DTMF. I suppose you can use a more forgiving
>     > >> discriminator if  all you need to tell is if you're
>     getting DTMF,
>     > >> but I've had a number  of real-world cases where wind
>     noise was
>     > >> detected as DTMF, and  there's always the ambiguity
>     when you have
>     > >> Captain Crunch on the  phone. In either case in the
>     non-optimized
>     > >> case it's the client who  gets to decide whether an
>     event should be
>     > >> interpreted as barge-in or  not, so it seems an
>     aesthetic protocol
>     > >> design decision whether the  client tells the server
>     ahead of time
>     > >> what circumstances to generate  the START-OF-SPEECH
>     event for, or
>     > >> whether the event gets generated  and the client
>     decides based on
>     > >> what's in the event whether it should  be
>     > treated
>     > >> as barge- in.
>     > >>
>     > >> I suppose one could make the argument that because
>     the spec implies
>     > >> that the event can only be generated once per request
>     that if a
>     > >> DTMF/ speech capable recognizer first hears
>     > enough noise
>     > >> to think it's  hearing speech and later hears DTMF,
>     the client will
>     > >> declare barge-in  when the event comes and not when he DTMF
>     > >> actually gets heard.
>     > >>
>     > >> If that's deemed a problem, we can still handle that
>     in the design
>     > >> where the server just reports what it's hearing by allowing
>     > >> multiple  events to be generated during a single request.
>     > >>
>     > >> Between the approach just outlined above, and an
>     approach where the
>     > >> client provides a filter for whether to generate the
>     event or not,
>     > >> I  have a mild preference (based on aesthetics rather
>     than some
>     > >> hard  engineering tradeoff) for the approach where
>     > the server
>     > >> just reports  what it's hearing.
>     > >>
>     > >>
>     > >> Having had some useful exchanges on this topic, it
>     also is becoming
>     > >> apparent to me that this event is poorly named, and
>     we should
>     > >> consider renaming it to "INTERESTING-INPUT-HEARD" or
>     something akin
>     > >> to that, because as others have pointed out, a
>     DTMF-only recognizer
>     > >> will never detect "start of speech".
>     > >>
>     > >> Another consideration to fold into the design choice is
>     > >> extensibility. Bear with me through a little
>     gedankenexperiment.
>     > >>
>     > >> Suppose we want to define a new recognizer type,
>     which I'll call
>     > >> the "name that tune" recognizer. The client plays
>     music to the
>     > >> server and  the server recognizes musical notes. The
>     grammar is a
>     > >> standard  musical notation, augmented with a semantic
>     > >> interpretation that  transforms the notes into the
>     title of the
>     > >> tune and provides that as  an answer.
>     > >>
>     > >> First, there's no speech involved (or is there...hang
>     on a minute).
>     > >> Second, in order to accommodate the "name that tune"
>     > >> recognizer, we'd have to extend both the client and
>     the server to
>     > >> undetstand a  directive as to whether to recognize
>     music or now,
>     > >> inaddition to what  the server already knows what to
>     do based on
>     > >> the grammar. If you  follow my logic above, whether
>     or not we do
>     > >> that, we have to extend  "start-of-speech" to say
>     "I'm hearing
>     > >> music". So far fairly  straightforward, but let me
>     now throw in the
>     > >> pathological twist.
>     > >>
>     > >> Suppose what I feed to a  combined music/speech
>     recognizer is a
>     > >> work  in sprechstimme (spoken music), like the
>     "Geographical Fugue"
>     > >> (aside:  this is a wonderful piece of music I highly
>     recommend to
>     > >> anyone  interested in small ensemble singing). In
>     this case, the
>     > >> tune could  be named by either doing speech or music
>     recognition.
>     > >> Why is there  any need for the client to constrain
>     the server as to
>     > >> which it tries  to do when it's
>     > already
>     > >> told the server what it wants through the  grammar?
>     > >>
>     > >> A few other comments below
>     > >>
>     > >>
>     > >>> ii. Flexibility: Want speech-only recognition
>     (because a second
>     > >>> recogniser is doing hotword on DTMF)
>     > >>>
>     > >>>
>     > >> I don't see how flexibility is affected by this deisgn
>     > choice. If
>     > >> that's what you want, feed the speech-only recognizer
>     a grammar
>     > >> without any DTMF rules.
>     > >>
>     > >>
>     > >>> + How to implement 5?
>     > >>>
>     > >>> a. Implicitly:
>     > >>>    - dtmfrecog: always DTMF-only recognition
>     > >>>    - speechrecog: depends on active grammar type
>     > >>>        > if a dtmf grammar is active then DTMF input is "on"
>     > >>>        > if a speech grammar is active then speech
>     input is "on"
>     > >>>
>     > >>> b. Explicitly:
>     > >>>    - Add inputmodes header to RECOGNIZE
>     > >>>
>     > >>> Option a is David's "do what I mean case"; option b is
>     > the extra
>     > >>> dial for the client.
>     > >>>
>     > >>>
>     > >> Actually, that's not the point I was making with "do
>     what I mean",
>     > >> but it's not essential to the discussion so let's move on.
>     > >>
>     > >>
>     > >>> It is worth noting that VoiceXML uses option b. This
>     allows one to
>     > >>> activate both speech grammars and DTMF grammars (and
>     > therefore
>     > >>> be informed of any errors in the grammars at activation
>     > time) but
>     > >>> independently turn on whichever input mode you like e.g.
>     > perhaps
>     > >>> start with "both" then change to "dtmf".
>     > >>>
>     > >>>
>     > >> I'm not sure the VXML precedent is relevant here,
>     because the
>     > >> application behind VXML is working a different part of the
>     > problem
>     > >> -  how to traverse a TUI dialog based on different
>     parts of the
>     > >> input  space. In fact, I suspect that the VXML: choice was
>     > >> conditioned more  by limitations at the time it was
>     > specified than
>     > >> an underlying good  design choice. Clearly having to
>     specify this
>     > >> in VXML make the job of  handling a TUI with nodes
>     like "Say or
>     > >> press 5" harder rather than  easier.
>     > >>
>     > >
>     > > DB> The VoiceXML edge-case is pretty weird so it's not a major
>     > > concern. My main concern is that the client can
>     indicate, somehow,
>     > > what the input modes are.
>     > >
>     > >
>     > >>
>     > >> Summing up, while I don't feel strongly one way or
>     the other, I
>     > >> have  a preference for handling this as follows:
>     > >>
>     > >> a) Rename "START-OF-SPEECH" to
>     "INTERESTING-INPUT-RECEIVED" or
>     > >> something equivalent.
>     > >> b) Include a parameter in the event saying what was
>     interesting
>     > >> about  the input you received, with a registry of values  
> which
>     > >> includes:
>     > >>     - signal above noise floor
>     > >>     - speech
>     > >>     - dtmf
>     > >>     - (possibly) music
>     > >> c) allow the event to be generated multiple times
>     during a request
>     > >>
>     > >>
>     > >
>     > > DB> I like these suggestions (START-OF-INPUT?).
>     However, I don't
>     > > see how the problem of the optimised case is not
>     solved by them. I
>     > > think the optimised case is fine if we have the
>     following rules:
>     > >
>     > Yes, I hadn't thought through the optimized case as
>     thoroughly as you.
>     > Your suggested method name is fine by me as well
>     >
>     > > 1. START-OF-SPEECH (and optimised bargin) is only generated
>     > for the
>     > > input type that is being listened for 2. A speechrecog
>     listens for
>     > > DTMF if DTMF grammars are active, speech if speech
>     grammars are
>     > > active, or speech and DTMF if both grammar types are active.
>     > >
>     > Works for me.
>     >
>     > >
>     > >> Note that all of the above I'm saying with my technical
>     > hat on and
>     > >> my chair hat off.
>     > >> Putting my chair hat on for a moment, we really need
>     to get this
>     > >> spec  to last call, so at some point Eric or I is going to
>     > declare
>     > >> rough  consensus so we can move on.
>     > >>
>     > >
>     > > DB> Agreed!
>     > >
>     > >
>     > >>
>     > >> Dave Oran.
>     > >>
>     > >>> Dave
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>> Sarvi makes a good point that adding the reason why the
>     > START-OF-
>     > >>> SPEECH occurred does not fix the optimised bargin case.
>     > >>>
>     > >>> dtmfrecog - listens for DTMF only
>     > >>> speechrecog - listens for DTMF only, or speech only,
>     or speech &
>     > >>> DTMF
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>>
>     > >>> ----- Original Message ----- From: "Shanmugham, Saravanan"
>     > >>> <[email-masked]>
>     > >>> To: "David R Oran" <[email-masked]>; "Dave Burke"
>     > >>> <[email-masked]>
>     > >>> Cc: <[email-masked]>; "Klaus Reifenrath"
>     > >>> <[email-masked]>
>     > >>> Sent: Tuesday, July 05, 2005 8:51 PM
>     > >>> Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode
>     > >>>
>     > >>>
>     > >>> inline.
>     > >>>
>     > >>>     -----Original Message-----
>     > >>>     From: David R Oran [mailto:[email-masked]]
>     > >>>     Sent: Tuesday, July 05, 2005 11:08 AM
>     > >>>     To: Dave Burke
>     > >>>     Cc: Shanmugham, Saravanan; Klaus Reifenrath;
>     [email-masked]
>     > >>>     Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only  
> mode
>     > >>>
>     > >>>
>     > >>>     On Jul 5, 2005, at 12:54 PM, Dave Burke wrote:
>     > >>>
>     > >>>     > Inline.
>     > >>>     >
>     > >>>     > Dave
>     > >>>     >
>     > >>>     > ----- Original Message ----- From:
>     "Shanmugham, Saravanan"
>     > >>>     > <[email-masked]>
>     > >>>     > To: "David R Oran" <[email-masked]>; "Klaus  
> Reifenrath"
>     > >>>     > <[email-masked]>
>     > >>>     > Cc: <[email-masked]>
>     > >>>     > Sent: Tuesday, July 05, 2005 5:22 PM
>     > >>>     > Subject: RE: [Speechsc] START-OF-SPEECH in
>     DTMF-only mode
>     > >>>     >
>     > >>>     >
>     > >>>     > I agree with Dave's analysis. The purpose of this  
> event
>     > >>>     was barge-in.
>     > >>>     > And barge-in should happen for both DTMF and speech.
>     > >>>     >
>     > >>>     > Is there a case where you think it should not behave
>     > >>>     this way. If soe,
>     > >>>     > please provide a scenario where you think
>     > >>>     >    1. Barge-in should happen for DTMF and not voice or
>     > >>>     vice-versa.
>     > >>>     >
>     > >>>     > DB> You want to do a DTMF recognition only
>     because it is
>     > >>> noisy.
>     > >>>     > While waiting for DTMF input, the speechrecog resource
>     > >>>     (or advanced
>     > >>>     > dtmfrecog) generates a START-OF-SPEECH because it  
> heard
>     > >>>     some speech.
>     > >>>     > The client does not want to stop prompt playing unless
>     > >>>     DTMF was heard
>     > >>>     > but it can't tell by the START-OF-SPEECH whether  
> speech
>     > >>>     or DTMF was
>     > >>>     > heard. Similarly vice versa.
>     > >>>     >
>     > >>>     It's an interesting design question what part of the
>     > >>>     policy resides at the client and what at the server, and
>     > >>>     who makes the "final decision" about whether what was
>     > >>>     heard was relevant to the control channel. Right now we
>     > >>>     (IMO) have a weird partitioning in many cases where the
>     > >>>     client basically says "do what I mean", but there are no
>     > >>>     constraints of what the server actually does, and no
>     > >>>     normalized basis for the client to figure out what to  
> set
>     > >>>     various magic numbers to (e.g. sensitivity).
>     > >>>
>     > >>>     In this case the only thing the client needs to  
> decide is
>     > >>>     whether to kill the prompt because the server thinks
>     > >>>     something that would interfere with the feedback
>     > >>>     ear/mouth/finger control happened. What this
>     says to me is
>     > >>>     that it isn't necessarily a good idea for the client to
>     > >>>     have more knobs to control the server
>     (especially if those
>     > >>>     knows are just more value/policy input ungrounded in any
>     > >>>     physics/ acoustics). On the other hand, having the  
> server
>     > >>>     tell the client more about what it thinks is going on is
>     > >>>     probably valuable.
>     > >>>
>     > >>>     So, Coming to the point after this long rambling
>     > >>>     introduction, I think it would in fact be useful for the
>     > >>>     START-Of-SPEECH event to indicate some extra  
> information,
>     > >>>     for example:
>     > >>>     a) I got something enough above the noise floor
>     to qualify
>     > >>>     for exceeding the "Sensisitvity" parameter you sent  
> in on
>     > >>>     the request but I really can't tell what it is
>     (could be a
>     > >>>     hippopatmus fart, or a siren in the background,
>     or captain
>     > >>>     crunch trying to whistle DTMF).
>     > >>>     b) I think I'm hearing speech
>     > >>>     c) I think I'm hearing DTMF
>     > >>>
>     > >>> Though I agree with your former part of your
>     response. I am not
>     > >>> sure I agree with your proposed solution.
>     > >>> The way I see this problem is that, it is more of what
>     > constitues a
>     > >>> barge-in event. This boils down to whether it is
>     speech, DTMF or
>     > >>> both.
>     > >>> This is inturn boils down to what type of recognizer
>     > resource we are
>     > >>> using, dtmf-recog, speech-recog, and speech-only-recog(we
>     > don't have
>     > >>> this and I don't think we should add it, but think
>     of this as a
>     > >>> place holder that explains the concept).
>     > >>>
>     > >>> A client knowing what type of barge-in happenned, does
>     > not impact
>     > >>> the
>     > >>> barge-in operation itself as it may be too late(for
>     the optimized
>     > >>> barge-in case). It may have other use cases, and if we
>     > can identify
>     > >>> them, I don't mind adding support for the
>     START-OF-SPEECH event to
>     > >>> say what type of barge-in happenned. But that itself does  
> not
>     > solve the
>     > >>> original problem raised. Refer to my previous response.
>     > >>>
>     > >>> The solution lies in defining what what is a
>     barge-in event.
>     > >>> That  boils
>     > >>> down to what type of recognition is happenning,
>     > dtmf-only, speech-
>     > >>> dtmf
>     > >>> or speech-only. We do not support speech-only as a
>     resource today,
>     > >>> the question is do we need a header to force it.
>     > >>>
>     > >>> Sarvi
>     > >>>
>     > >>>
>     > >>>     >    2. You would benefit from the client knowing
>     > what caused
>     > >>> the
>     > >>>     > barge-in, DTMF Vs speech.
>     > >>>     >
>     > >>>     > DB> See previous comment. And previous e-mail:
>     either add an
>     > >>>     > inputmodes header (taking value speech, dtmf, both) to
>     > >>>     the RECOGNIZE
>     > >>>     > request or add a header to the START-OF-SPEECH event
>     > >>>     indicating DTMF
>     > >>>     > or speech.
>     > >>>     >
>     > >>>     I'm leaning in your direction on this latter point - as
>     > >>>     should be evident from what I wrote above.
>     > >>>
>     > >>>     > Sarvi
>     > >>>     >
>     > >>>     >     -----Original Message-----
>     > >>>     >     From: [email-masked]
>     > >>>     >     [mailto:[email-masked]] On Behalf Of
>     > David R
>     > >>> Oran
>     > >>>     >     Sent: Tuesday, July 05, 2005 5:24 AM
>     > >>>     >     To: Klaus Reifenrath
>     > >>>     >     Cc: [email-masked]'
>     > >>>     >     Subject: Re: [Speechsc] START-OF-SPEECH in
>     > DTMF-only mode
>     > >>>     >
>     > >>>     >
>     > >>>     >     On Jul 5, 2005, at 3:46 AM, Reifenrath,
>     Klaus wrote:
>     > >>>     >
>     > >>>     >     > The current spec is not clear when
>     > START-OF-SPEECH need
>     > >>>     >     to be send in
>     > >>>     >     > the following scenarios:
>     > >>>     >     > A) The client requested a DTMF Recognizer. Is  
> the
>     > >>>     >     START-OF-SPEECH
>     > >>>     >     > event send to the client also if speech
>     was detected?
>     > >>>     >     I suspect so, since one of the prime purposes
>     > is to enable
>     > >>>     >     client- mediated barge-in handling. However, if  
> the
>     > >>>     >     recognizer is in fact only capable of
>     recognizing DTMF
>     > >>>     >     then it may in fact not report anythin
>     unless it's using
>     > >>>     >     some primitive thresholding machinery, like a SN
>     > >>> threshold.
>     > >>>     >     > B) The client requested a Speech
>     Recognizer, but only
>     > >>>     >     activated DTMF
>     > >>>     >     > grammars. Is the START-OF-SPEECH event
>     send to the
>     > >>>     >     client also if
>     > >>>     >     > speech was detected?
>     > >>>     >     Again, I'd say yes, for the same reason as above.
>     > >>>     >     > I think in both cases START-OF-SPEECH should  
> only
>     > >>>     be send after
>     > >>>     >     > detecting a DTMF digit (see Figure 12 of  
> VoiceXML
>     > >>>     2.0: http://
>     > >>>     >     > www.w3.org/TR/voicexml20/#dmlATiming).
>     > >>>     >     We seem to have reached different
>     conclusions. I'd be
>     > >>>     >     interested in why you think my analysis
>     above is wrong.
>     > >>>     >
>     > >>>     >     Dave.
>     > >>>     >
>     > >>>     >     > Klaus
>     > >>>     >     >
>     > >>>     >     > _______________________________________________
>     > >>>     >     > Speechsc mailing list
>     > >>>     >     > [email-masked]
>     > >>>     >     > https://www1.ietf.org/mailman/listinfo/speechsc
>     > >>>     >     >
>     > >>>     >
>     > >>>     >     _______________________________________________
>     > >>>     >     Speechsc mailing list
>     > >>>     >     [email-masked]
>     > >>>     >     https://www1.ietf.org/mailman/listinfo/speechsc
>     > >>>     >
>     > >>>     >
>     > >>>     > _______________________________________________
>     > >>>     > Speechsc mailing list
>     > >>>     > [email-masked]
>     > >>>     > https://www1.ietf.org/mailman/listinfo/speechsc
>     > >>>     >
>     > >>>
>     > >>>
>     > >>> _______________________________________________
>     > >>> Speechsc mailing list
>     > >>> [email-masked]
>     > >>> https://www1.ietf.org/mailman/listinfo/speechsc
>     > >>>
>     > >>>
>     > >>
>     > >> _______________________________________________
>     > >> Speechsc mailing list
>     > >> [email-masked]
>     > >> https://www1.ietf.org/mailman/listinfo/speechsc
>     > >
>     >
>     > _______________________________________________
>     > Speechsc mailing list
>     > [email-masked]
>     > https://www1.ietf.org/mailman/listinfo/speechsc
>     >
>
>
>     _______________________________________________
>     Speechsc mailing list
>     [email-masked]
>     https://www1.ietf.org/mailman/listinfo/speechsc
>
>
> _______________________________________________
> Speechsc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/speechsc
>
>
> _______________________________________________
> Speechsc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/speechsc
>

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: David R Oran <[email-masked]>
Date: Mon, 11 Jul 2005 09:15:28 -0400

> you will need to document the behavior for inconsistencies.
Right. I generally agree with Pierre, which is why I proposed a  
compromise position where the client can use a header to specify what  
types of input should be considered the start of something  
potentially interesting from a barge-in point of view, as opposed to  
a directive to the server for what to recognize.

I don't think this is crucial functionality, but it does make for  
more consistent behavior between the optimized and non-optimized  
barge-in cases. In fact, I can see cases where the client doesn't  
want any barge-in processing to happen. Imagine an application called  
"mark the musical phrases", where the client listens to playout of a  
musical score, and hits various DTMF buttons to indicate markers  
(e.g. end-of-phrase, end-of-theme, key-change) while listening. For  
this application you want to make sure that barge-in doesn't stop the  
music!

> Pmf> I agree we should not have multiple SOS events.  Too much  
I'm not concerned about the "chatter" since this is a low-bandwidth  
control channel even with multiple events, but as I said it's a small  
point and I'm happy to concede.

> clear reason for changing - which I do not see here.
I do see a clear reason, since the thing that you see the start of  
may not be speech. I like (I think it was Dave's suggestion) "start- 
of-input:" since it mirrors the other method names.
> definitely make this optional
I hope I explained the intent above. This header would not define the  
input types that the recognize would process, but rather the input  
types the client wants it to consider potential barge-in and hence  
generate the "start-of-input:" event for and do any local barge-in  
optimizations for.

Dave.

---

From: "Dave Burke" <[email-masked]>
Date: Mon, 11 Jul 2005 22:21:25 +0100

I don't think it is possible in practice to separate what input type is 
considered potential barge-in and what input type is being recognised 
because a recognition hypothesis is always generated when a barge-in occurs. 
For example, imagine a client invoked a speechrecog and asked it to only 
consider DTMF for barge-in. Once speech is detected, although no barge-in 
would happen, a RECOGNITION-COMPLETE will result after 
Speech-Incomplete-Timeout milliseconds of silence thus ending the 
recognition state machine with a 'nomatch'. The converse example applies for 
a speech-only recognition (DTMF-Interdigit-Timeout replaces the 
Speech-Incomplete-Timeout).

Although I prefer the idea of an InputModes header in RECOGNIZE (it's the 
cleanest solution for VoiceXML implementors), I am willing to compromise on 
using the grammar type to determine the input types (in what follows I use 
input types to mean both what is considered potential barge-in and what type 
of input is to be recognised). The main issue with this approach is 
incompatibility with VoiceXML (recall VoiceXML grammar activation is 
independent of what inputmodes are set).

This incompatibility results in side-effects that, in my opinion, are 
inconsequential for real applications:
    a. speech barge-in (noise) and nomatch will not happen for 
inputmodes="both" when no speech grammars are activated
    b. dtmf barge-in and nomatch will not happen for inputmodes="both" when 
no dtmf grammars are activated
    c. grammars with errors will not be detected if the grammar type is not 
compatible with the inputmodes property
A workaround for the really conscientious VoiceXML platform is to activate a 
dummy grammar (maybe with the NULL rule) of type speech (dtmf) when 
inputmodes="both" is set but no speech (dtmf) grammar is activated in the 
application.

-----------------

Summarising the proposed changes again  (refined a little and to avoid 
trawling though this massive thread!):

1. For a speechrecog resource, the types of the activated grammars determine 
the input types the recogniser processes and considers for potential 
barge-in.

2. Clarify a dtmfrecog only processes DTMF and hence can only generate 
barge-in / START-OF-SPEECH for DTMF inputs

Nice-to-have:

3. Change START-OF-SPEECH to START-OF-INPUT

4. Add a header of InputType to START-OF-INPUT. Current specified values are 
"dtmf" or "speech".

-----------------

Dave

----- Original Message ----- 
From: "David R Oran" <[email-masked]>
To: "Pierre Forgues" <[email-masked]>
Cc: <[email-masked]>; "Shanmugham, Saravanan" <[email-masked]>; "Eric 
Burger" <[email-masked]>; "Dave Burke" <[email-masked]>
Sent: Monday, July 11, 2005 2:15 PM
Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?)

>> Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary (?)
>>
>>
> types of input should be considered the start of something  potentially 
> interesting from a barge-in point of view, as opposed to  a directive to 
> the server for what to recognize.
>
> I don't think this is crucial functionality, but it does make for  more 
> consistent behavior between the optimized and non-optimized  barge-in 
> cases. In fact, I can see cases where the client doesn't  want any 
> barge-in processing to happen. Imagine an application called  "mark the 
> musical phrases", where the client listens to playout of a  musical score, 
> and hits various DTMF buttons to indicate markers  (e.g. end-of-phrase, 
> end-of-theme, key-change) while listening. For  this application you want 
> to make sure that barge-in doesn't stop the  music!
>
>>
> I do see a clear reason, since the thing that you see the start of  may 
> not be speech. I like (I think it was Dave's suggestion) "start- 
> input types that the recognize would process, but rather the input  types 
> the client wants it to consider potential barge-in and hence  generate the 
> "start-of-input:" event for and do any local barge-in  optimizations for.
>
>
> Dave.
>
>>

---

From: David R Oran <[email-masked]>
Date: Wed, 13 Jul 2005 09:20:22 -0400

On Jul 11, 2005, at 5:21 PM, Dave Burke wrote:

> I don't think it is possible in practice to separate what input  
> type is considered potential barge-in and what input type is being  
> recognised because a recognition hypothesis is always generated  
> when a barge-in occurs.
True, but what I'm trying to tease apart is whether to declare barge- 
in from the point of view of stopping output from simply detecting  
input for the purposes of cranking up the non-signal-processing parts  
of the recognition engine. Did you not find my "mark the phrases"  
application example compelling?

> For example, imagine a client invoked a speechrecog and asked it to  
> only consider DTMF for barge-in. Once speech is detected, although  
> no barge-in would happen, a RECOGNITION-COMPLETE will result after  
> recognition state machine with a 'nomatch'.
Sure, but of course the timeout could be very long... I must be  
dense, because I see this relationship as tenuous rather than  
strongly coupled.

> The converse example applies for a speech-only recognition (DTMF- 
> Interdigit-Timeout replaces the Speech-Incomplete-Timeout).
>
> Although I prefer the idea of an InputModes header in RECOGNIZE  
> (it's the cleanest solution for VoiceXML implementors), I am  
> willing to compromise on using the grammar type to determine the  
> input types (in what follows I use input types to mean both what is  
> considered potential barge-in and what type of input is to be  
> recognised). The main issue with this approach is incompatibility  
> with VoiceXML (recall VoiceXML grammar activation is independent of  
> what inputmodes are set).
>
And voice XML does not have any explicit control over barge-in  
either. We could of course back off not have clients involved at all  
in barge-in processing, but I think that would be a mistake.

> This incompatibility results in side-effects that, in my opinion,  
> are inconsequential for real applications:
>    b. dtmf barge-in and nomatch will not happen for  
> inputmodes="both" when no dtmf grammars are activated
>    c. grammars with errors will not be detected if the grammar type  
> is not compatible with the inputmodes property
> A workaround for the really conscientious VoiceXML platform is to  
> activate a dummy grammar (maybe with the NULL rule) of type speech  
> (dtmf) when inputmodes="both" is set but no speech (dtmf) grammar  
> is activated in the application.
>
I agree with this assessment. I'll also point out that the  
flexibility will allow a class of VXML applications that can't be  
done today (e.g. ones that don't stop prompts on hearing input.

> -----------------
>
> Summarising the proposed changes again  (refined a little and to  
> avoid trawling though this massive thread!):
>
> 1. For a speechrecog resource, the types of the activated grammars  
> determine the input types the recogniser processes and considers  
> for potential barge-in.
>
Good. Agree.

> 2. Clarify a dtmfrecog only processes DTMF and hence can only  
> generate barge-in / START-OF-SPEECH for DTMF inputs
>
I'm ok with this even though I'd rather keep input detection and  
recognition behavior orthogonal to accommodate smarter server  
implementations and to give better consistency between the server- 
optimized and client-intermediated cases. In particular, I do really  
feel like we need a way for the client to tell a server to NOT try to  
outguess it by doing optimized barge in processing since the  
application might in fact not want the output stopped.

> 3. Change START-OF-SPEECH to START-OF-INPUT
I'm in favor of this.

> values are "dtmf" or "speech".
I think we really need this, and would press again that we make this  
a list of values and future-proof it with the other values we suspect  
will be quite useful (e.g. music, gesture).

Dave. (chair hat sort-of-on since we're trying to get final consensus  
here).

> -----------------
>
> Dave
>
> ----- Original Message ----- From: "David R Oran" <[email-masked]>
> Cc: <[email-masked]>; "Shanmugham, Saravanan" <[email-masked]>;  
> "Eric Burger" <[email-masked]>; "Dave Burke"  
>>> summary (?)
>> compromise position where the client can use a header to specify  
>> what types of input should be considered the start of something   
>> potentially interesting from a barge-in point of view, as opposed  
>> to  a directive to the server for what to recognize.
>>
>> want any barge-in processing to happen. Imagine an application  
>> called  "mark the musical phrases", where the client listens to  
>> playout of a  musical score, and hits various DTMF buttons to  
>> indicate markers  (e.g. end-of-phrase, end-of-theme, key-change)  
>> while listening. For  this application you want to make sure that  
>> barge-in doesn't stop the  music!
>>
>>
>>>
>>>
>> I'm not concerned about the "chatter" since this is a low- 
>> bandwidth control channel even with multiple events, but as I said  
>> it's a small point and I'm happy to concede.
>>
>>
>>>
>>>
>>>
>> I do see a clear reason, since the thing that you see the start  
>> of  may not be speech. I like (I think it was Dave's suggestion)  
>> "start- of-input:" since it mirrors the other method names.
>>
>>>
>>>
>> I hope I explained the intent above. This header would not define  
>> the input types that the recognize would process, but rather the  
>> input  types the client wants it to consider potential barge-in  
>> and hence  generate the "start-of-input:" event for and do any  
>> local barge-in  optimizations for.
>>
>>
>> Dave.
>>
>>
>>>
>>>

---

From: "Dave Burke" <[email-masked]>
Date: Wed, 13 Jul 2005 17:07:19 +0100

Inline

Dave

----- Original Message ----- 
From: "David R Oran" <[email-masked]>
To: "Dave Burke" <[email-masked]>
Cc: "Pierre Forgues" <[email-masked]>; <[email-masked]>; "Shanmugham, 
Saravanan" <[email-masked]>; "Eric Burger" <[email-masked]>
Sent: Wednesday, July 13, 2005 2:20 PM
Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?)

> True, but what I'm trying to tease apart is whether to declare barge- in 
> from the point of view of stopping output from simply detecting  input for 
> the purposes of cranking up the non-signal-processing parts  of the 
> recognition engine. Did you not find my "mark the phrases"  application 
> example compelling?

DB> Indeed it's a nice application (albeit producing disturbing memories of 
musical theory exams from my distant childhood!)... Can't this application 
be created by setting Kill-On-Barge-In to false on the SPEAK request that 
queued the music?

>> consider DTMF for barge-in. Once speech is detected, although  no 
>> barge-in would happen, a RECOGNITION-COMPLETE will result after 
> Sure, but of course the timeout could be very long... I must be  dense, 
> because I see this relationship as tenuous rather than  strongly coupled.

DB> But the recognition will terminate with nomatch at some point if it 
hears noise/speech first regardless of a good DTMF sequence (I say first 
only out of pragmatism because that's what modern day recognizers do - the 
first input type recognized results in turning off the other input type for 
the remainder of the recognition). The recognition will need to be restarted 
but then some DTMF is lost, timers have to be reset etc.

>> Although I prefer the idea of an InputModes header in RECOGNIZE  (it's 
>> the cleanest solution for VoiceXML implementors), I am  willing to 
>> compromise on using the grammar type to determine the  input types (in 
>> what follows I use input types to mean both what is  considered potential 
>> barge-in and what type of input is to be  recognised). The main issue 
>> with this approach is incompatibility  with VoiceXML (recall VoiceXML 
>> grammar activation is independent of  what inputmodes are set).
>>
> And voice XML does not have any explicit control over barge-in  either. We 
> could of course back off not have clients involved at all  in barge-in 
> processing, but I think that would be a mistake.

DB> I'm not sure I understand this (unless you mean VoiceXML doesn't 
decouple barge-in input type from recognition input type in which case I 
agree). VoiceXML allows you to switch on/off barge-in by setting the 
barge-in attribute on <prompt> (maps to MRCP's Kill-On-Barge-In). It allows 
you to simultaneously set the input type for recognition and barge-in. 
VoiceXML does have a bargeintype attribute although this is just to turn on 
hotword...

>>    b. dtmf barge-in and nomatch will not happen for  inputmodes="both" 
>> when no dtmf grammars are activated
>>    c. grammars with errors will not be detected if the grammar type  is 
>> not compatible with the inputmodes property
>> (dtmf) when inputmodes="both" is set but no speech (dtmf) grammar  is 
>> activated in the application.
>>
> I agree with this assessment. I'll also point out that the  flexibility 
> will allow a class of VXML applications that can't be  done today (e.g. 
> ones that don't stop prompts on hearing input.

DB> This is possible already with Kill-On-Barge-On false

>> determine the input types the recogniser processes and considers  for 
>> potential barge-in.
>>
> Good. Agree.
>
> optimized and client-intermediated cases. In particular, I do really  feel 
> like we need a way for the client to tell a server to NOT try to  outguess 
> it by doing optimized barge in processing since the  application might in 
> fact not want the output stopped.
>
>
>> Nice-to-have:
>>
>> 4. Add a header of InputType to START-OF-INPUT. Current specified  values 
>> are "dtmf" or "speech".
>>
> I think we really need this, and would press again that we make this  a 
> list of values and future-proof it with the other values we suspect  will 
> be quite useful (e.g. music, gesture).
>
>>>> From: [email-masked] [mailto:speechsc- [email-masked]] On
>>>> Behalf Of David R Oran
>>> interesting from a barge-in point of view, as opposed  to  a directive 
>>> to the server for what to recognize.
>>>
>>> musical phrases", where the client listens to  playout of a  musical 
>>> score, and hits various DTMF buttons to  indicate markers  (e.g. 
>>> end-of-phrase, end-of-theme, key-change)  while listening. For  this 
>>> application you want to make sure that  barge-in doesn't stop the 
>>> music!
>>>
>>>
>>>>
>>>>
>>> I'm not concerned about the "chatter" since this is a low- bandwidth 
>>>> Ok, with slight modification. Say that the syntax of the "input- type"

---

From: David R Oran <[email-masked]>
Date: Wed, 13 Jul 2005 12:54:40 -0400

On Jul 13, 2005, at 12:07 PM, Dave Burke wrote:

> Cc: "Pierre Forgues" <[email-masked]>; <[email-masked]>;  
> "Shanmugham, Saravanan" <[email-masked]>; "Eric Burger"  
> <[email-masked]>
>>> type is considered potential barge-in and what input type is  
>>> being  recognised because a recognition hypothesis is always  
>>> generated  when a barge-in occurs.
>>>
>> True, but what I'm trying to tease apart is whether to declare  
>> barge- in from the point of view of stopping output from simply  
>> detecting  input for the purposes of cranking up the non-signal- 
>> processing parts  of the recognition engine. Did you not find my  
>> "mark the phrases"  application example compelling?
>>
>
> DB> Indeed it's a nice application (albeit producing disturbing  
> memories of musical theory exams from my distant childhood!)...  
> Can't this application be created by setting Kill-On-Barge-In to  
> false on the SPEAK request that queued the music?
>
Ah, true if the output was from a SPEAK request. Not sure if we need  
to accommodate other cases.

>
>>
>>
>>> For example, imagine a client invoked a speechrecog and asked it  
>>> to  only consider DTMF for barge-in. Once speech is detected,  
>>> although  no barge-in would happen, a RECOGNITION-COMPLETE will  
>>> result after Speech-Incomplete-Timeout milliseconds of silence  
>>> thus ending the recognition state machine with a 'nomatch'.
>>>
> DB> But the recognition will terminate with nomatch at some point  
> if it hears noise/speech first regardless of a good DTMF sequence  
> (I say first only out of pragmatism because that's what modern day  
> recognizers do - the first input type recognized results in turning  
> off the other input type for the remainder of the recognition). The  
> recognition will need to be restarted but then some DTMF is lost,  
> timers have to be reset etc.
>
Yes, it's a tradeoff between pragmatic stuff and clean aesthetics and  
abstraction. I tend to favor the latter, all things being equal  
(which they rarely are).
>>> input types (in what follows I use input types to mean both what  
>>> is  considered potential barge-in and what type of input is to  
>>> be  recognised). The main issue with this approach is  
>>> incompatibility  with VoiceXML (recall VoiceXML grammar  
>>> activation is independent of  what inputmodes are set).
>>>
>>>
>> either. We could of course back off not have clients involved at  
>> all  in barge-in processing, but I think that would be a mistake.
>>
>
> DB> I'm not sure I understand this (unless you mean VoiceXML  
> doesn't decouple barge-in input type from recognition input type in  
> which case I agree).
Yes, that's what I meant.

> VoiceXML allows you to switch on/off barge-in by setting the barge- 
> in attribute on <prompt> (maps to MRCP's Kill-On-Barge-In). It  
> allows you to simultaneously set the input type for recognition and  
> barge-in. VoiceXML does have a bargeintype attribute although this  
> is just to turn on hotword...
>
Which seems another mixing of fruit in the fruit salad :-)

>
>>
>>
>>> This incompatibility results in side-effects that, in my  
>>> opinion,  are inconsequential for real applications:
>>>    c. grammars with errors will not be detected if the grammar  
>>> type  is not compatible with the inputmodes property
>>> activate a dummy grammar (maybe with the NULL rule) of type  
>>> speech (dtmf) when inputmodes="both" is set but no speech (dtmf)  
>>> grammar  is activated in the application.
>>>
>>>
Ok, I concede the point. You're right.
>>> 1. For a speechrecog resource, the types of the activated  
>>> grammars determine the input types the recogniser processes and  
>>> considers  for potential barge-in.
>>>
>>>
>> Good. Agree.
>>
>>
>> optimized and client-intermediated cases. In particular, I do  
>> really  feel like we need a way for the client to tell a server to  
>> NOT try to  outguess it by doing optimized barge in processing  
>> since the  application might in fact not want the output stopped.
>>
>>
>>
>>> Nice-to-have:
>>>
>>> 4. Add a header of InputType to START-OF-INPUT. Current  
>>> specified  values are "dtmf" or "speech".
>>>
>>>
>> I think we really need this, and would press again that we make  
>> this  a list of values and future-proof it with the other values  
>> we suspect  will be quite useful (e.g. music, gesture).
>>
>> Dave. (chair hat sort-of-on since we're trying to get final  
>> consensus here).
>>
>>
>>
>>> -----------------
>>>
>>> Dave
>>>
>>> <[email-masked]>; "Eric Burger" <[email-masked]>; "Dave  
>>> Burke" <[email-masked]>
>>>> compromise position where the client can use a header to  
>>>> specify  what types of input should be considered the start of  
>>>> something   potentially interesting from a barge-in point of  
>>>> view, as opposed  to  a directive to the server for what to  
>>>> recognize.
>>>>
>>>> I don't think this is crucial functionality, but it does make  
>>>> for   more consistent behavior between the optimized and non- 
>>>> optimized   barge-in cases. In fact, I can see cases where the  
>>>> client doesn't   want any barge-in processing to happen. Imagine  
>>>> an application  called  "mark the musical phrases", where the  
>>>> client listens to  playout of a  musical score, and hits various  
>>>> DTMF buttons to  indicate markers  (e.g. end-of-phrase, end-of- 
>>>> theme, key-change)  while listening. For  this application you  
>>>> want to make sure that  barge-in doesn't stop the music!
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>> bandwidth control channel even with multiple events, but as I  
>>>> said  it's a small point and I'm happy to concede.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> of  may not be speech. I like (I think it was Dave's  
>>>> suggestion)  "start- of-input:" since it mirrors the other  
>>>> method names.
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>> I hope I explained the intent above. This header would not  
>>>> define  the input types that the recognize would process, but  
>>>> rather the  input types the client wants it to consider  
>>>> potential barge-in  and hence generate the "start-of-input:"  
>>>> event for and do any  local barge-in optimizations for.
>>>>
>>>>
>>>> Dave.
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>