Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Tue, 5 Jul 2005 12:51:57 -0700

inline.

     -----Original Message-----
     From: David R Oran [mailto:[email-masked]] 
     Sent: Tuesday, July 05, 2005 11:08 AM
     To: Dave Burke
     Cc: Shanmugham, Saravanan; Klaus Reifenrath; [email-masked]
     Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode

     On Jul 5, 2005, at 12:54 PM, Dave Burke wrote:
     
     > Inline.
     >
     > Dave
     >
     > ----- Original Message ----- From: "Shanmugham, Saravanan"  
     > <[email-masked]>
     > To: "David R Oran" <[email-masked]>; "Klaus Reifenrath"  
     > <[email-masked]>
     > Cc: <[email-masked]>
     > Sent: Tuesday, July 05, 2005 5:22 PM
     > Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode
     >
     >
     > I agree with Dave's analysis. The purpose of this event 
     was barge-in.
     > And barge-in should happen for both DTMF and speech.
     >
     > Is there a case where you think it should not behave 
     this way. If soe, 
     > please provide a scenario where you think
     >    1. Barge-in should happen for DTMF and not voice or 
     vice-versa.
     >
     > DB> You want to do a DTMF recognition only because it is noisy.  
     > While waiting for DTMF input, the speechrecog resource 
     (or advanced
     > dtmfrecog) generates a START-OF-SPEECH because it heard 
     some speech. 
     > The client does not want to stop prompt playing unless 
     DTMF was heard 
     > but it can't tell by the START-OF-SPEECH whether speech 
     or DTMF was 
     > heard. Similarly vice versa.
     >
     It's an interesting design question what part of the 
     policy resides at the client and what at the server, and 
     who makes the "final decision" about whether what was 
     heard was relevant to the control channel. Right now we 
     (IMO) have a weird partitioning in many cases where the 
     client basically says "do what I mean", but there are no 
     constraints of what the server actually does, and no 
     normalized basis for the client to figure out what to set 
     various magic numbers to (e.g. sensitivity).
     
     In this case the only thing the client needs to decide is 
     whether to kill the prompt because the server thinks 
     something that would interfere with the feedback 
     ear/mouth/finger control happened. What this says to me is 
     that it isn't necessarily a good idea for the client to 
     have more knobs to control the server (especially if those 
     knows are just more value/policy input ungrounded in any 
     physics/ acoustics). On the other hand, having the server 
     tell the client more about what it thinks is going on is 
     probably valuable.
     
     So, Coming to the point after this long rambling 
     introduction, I think it would in fact be useful for the 
     START-Of-SPEECH event to indicate some extra information, 
     for example:
     a) I got something enough above the noise floor to qualify 
     for exceeding the "Sensisitvity" parameter you sent in on 
     the request but I really can't tell what it is (could be a 
     hippopatmus fart, or a siren in the background, or captain 
     crunch trying to whistle DTMF).
     b) I think I'm hearing speech
     c) I think I'm hearing DTMF

Though I agree with your former part of your response. I am not sure I
agree with your proposed solution. 
The way I see this problem is that, it is more of what constitues a
barge-in event. This boils down to whether it is speech, DTMF or both.
This is inturn boils down to what type of recognizer resource we are
using, dtmf-recog, speech-recog, and speech-only-recog(we don't have
this and I don't think we should add it, but think of this as a place
holder that explains the concept).

A client knowing what type of barge-in happenned, does not impact the
barge-in operation itself as it may be too late(for the optimized
barge-in case). It may have other use cases, and if we can identify
them, I don't mind adding support for the START-OF-SPEECH event to say
what type of barge-in happenned. But that itself does not solve the
original problem raised. Refer to my previous response.

The solution lies in defining what what is a barge-in event. That boils
down to what type of recognition is happenning, dtmf-only, speech-dtmf
or speech-only. We do not support speech-only as a resource today, the
question is do we need a header to force it. 

Sarvi     

     >    2. You would benefit from the client knowing what caused the 
     > barge-in, DTMF Vs speech.
     >
     > DB> See previous comment. And previous e-mail: either add an
     > inputmodes header (taking value speech, dtmf, both) to 
     the RECOGNIZE 
     > request or add a header to the START-OF-SPEECH event 
     indicating DTMF 
     > or speech.
     I'm leaning in your direction on this latter point - as 
     should be evident from what I wrote above.
     
     > Sarvi
     >
     >     -----Original Message-----
     >     From: [email-masked]
     >     [mailto:[email-masked]] On Behalf Of David R Oran
     >     Sent: Tuesday, July 05, 2005 5:24 AM
     >     To: Klaus Reifenrath
     >     Cc: [email-masked]'
     >     Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode
     >
     >
     >     On Jul 5, 2005, at 3:46 AM, Reifenrath, Klaus wrote:
     >
     >     > The current spec is not clear when START-OF-SPEECH need
     >     to be send in
     >     > the following scenarios:
     >     > A) The client requested a DTMF Recognizer. Is the
     >     START-OF-SPEECH
     >     > event send to the client also if speech was detected?
     >     I suspect so, since one of the prime purposes is to enable
     >     client- mediated barge-in handling. However, if the
     >     recognizer is in fact only capable of recognizing DTMF
     >     then it may in fact not report anythin unless it's using
     >     some primitive thresholding machinery, like a SN threshold.
     >     > B) The client requested a Speech Recognizer, but only
     >     activated DTMF
     >     > grammars. Is the START-OF-SPEECH event send to the
     >     client also if
     >     > speech was detected?
     >     Again, I'd say yes, for the same reason as above.
     >     > I think in both cases START-OF-SPEECH should only 
     be send after
     >     > detecting a DTMF digit (see Figure 12 of VoiceXML 
     2.0: http://
     >     > www.w3.org/TR/voicexml20/#dmlATiming).
     >     We seem to have reached different conclusions. I'd be
     >     interested in why you think my analysis above is wrong.
     >
     >     Dave.
     >
     >     > Klaus
     >     >
     >     > _______________________________________________
     >     > Speechsc mailing list
     >     > [email-masked]
     >     > https://www1.ietf.org/mailman/listinfo/speechsc
     >     >
     >
     >     _______________________________________________
     >     Speechsc mailing list
     >     [email-masked]
     >     https://www1.ietf.org/mailman/listinfo/speechsc
     >
     >
     > _______________________________________________
     > Speechsc mailing list
     > [email-masked]
     > https://www1.ietf.org/mailman/listinfo/speechsc
     >

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: "Dave Burke" <[email-masked]>
Date: Wed, 6 Jul 2005 10:31:51 +0100

+ Attempting to summarise:

1. START-OF-SPEECH is useful for the client to know when to stop playing 
prompts the in non-optimised case
2. START-OF-SPEECH is useful for the client to calculate the bargin time 
(e.g. VoiceXML 2.1 <mark>)
3. In the optimised case, a bargin automatically stops prompt playing 
(assuming prompts barginable)
4. Because of the previous point, the question of what input type caused 
bargin is different and less important to what input type(s) the recogniser 
is listening for
5. Currently, MRCPv2 has no way of indicating what input type(s) a 
recogniser is listening for

+ Why implement 5?

i. Noisy case: Need DTMF-only recognition (and may only have a speechrecog)
ii. Flexibility: Want speech-only recognition (because a second recogniser 
is doing hotword on DTMF)

+ How to implement 5?

a. Implicitly:
    - dtmfrecog: always DTMF-only recognition
    - speechrecog: depends on active grammar type
        > if a dtmf grammar is active then DTMF input is "on"
        > if a speech grammar is active then speech input is "on"

b. Explicitly:
    - Add inputmodes header to RECOGNIZE

Option a is David's "do what I mean case"; option b is the extra dial for 
the client.

It is worth noting that VoiceXML uses option b. This allows one to activate 
both speech grammars and DTMF grammars (and therefore be informed of any 
errors in the grammars at activation time) but independently turn on 
whichever input mode you like e.g. perhaps start with "both" then change to 
"dtmf".

Dave

Sarvi makes a good point that adding the reason why the START-OF-SPEECH 
occurred does not fix the optimised bargin case.

dtmfrecog - listens for DTMF only
speechrecog - listens for DTMF only, or speech only, or speech & DTMF

----- Original Message ----- 
From: "Shanmugham, Saravanan" <[email-masked]>
To: "David R Oran" <[email-masked]>; "Dave Burke" <[email-masked]>
Cc: <[email-masked]>; "Klaus Reifenrath" <[email-masked]>
Sent: Tuesday, July 05, 2005 8:51 PM
Subject: RE: [Speechsc] START-OF-SPEECH in DTMF-only mode

inline.

     -----Original Message-----
     From: David R Oran [mailto:[email-masked]]
     Sent: Tuesday, July 05, 2005 11:08 AM
     To: Dave Burke
     Cc: Shanmugham, Saravanan; Klaus Reifenrath; [email-masked]
     Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode

     was barge-in.
     this way. If soe,
     vice-versa.
     (or advanced
     some speech.
     DTMF was heard
     or DTMF was

Sarvi

     the RECOGNIZE
     indicating DTMF

     be send after
     2.0: http://

---

From: David R Oran <[email-masked]>
Date: Wed, 6 Jul 2005 08:07:37 -0400

On Jul 6, 2005, at 5:31 AM, Dave Burke wrote:

> 1. START-OF-SPEECH is useful for the client to know when to stop  
> playing prompts the in non-optimised case
> 2. START-OF-SPEECH is useful for the client to calculate the bargin  
> time (e.g. VoiceXML 2.1 <mark>)
> 3. In the optimised case, a bargin automatically stops prompt  
> playing (assuming prompts barginable)
> 4. Because of the previous point, the question of what input type  
> caused bargin is different and less important to what input type(s)  
> the recogniser is listening for
I'm not sure I follow point 4. Could you elaborate?

> recogniser is listening for
Do you mean exactly this, or do you mean "for the client to indicate  
to the resource what input types it should look for"?

> i. Noisy case: Need DTMF-only recognition (and may only have a  
I'm having difficulty following the logic of why noise would  
necessarily trigger START-OF-SPEECH if you were listening for speech  
but not DTMF. I suppose you can use a more forgiving discriminator if  
all you need to tell is if you're getting DTMF, but I've had a number  
of real-world cases where wind noise was detected as DTMF, and  
there's always the ambiguity when you have Captain Crunch on the  
phone. In either case in the non-optimized case it's the client who  
gets to decide whether an event should be interpreted as barge-in or  
not, so it seems an aesthetic protocol design decision whether the  
client tells the server ahead of time what circumstances to generate  
the START-OF-SPEECH event for, or whether the event gets generated  
and the client decides based on what's in the event whether it should  
be treated as barge- in.

I suppose one could make the argument that because the spec implies  
that the event can only be generated once per request that if a DTMF/ 
speech capable recognizer first hears enough noise to think it's  
hearing speech and later hears DTMF, the client will declare barge-in  
when the event comes and not when he DTMF actually gets heard.

If that's deemed a problem, we can still handle that in the design  
where the server just reports what it's hearing by allowing multiple  
events to be generated during a single request.

Between the approach just outlined above, and an approach where the  
client provides a filter for whether to generate the event or not, I  
have a mild preference (based on aesthetics rather than some hard  
engineering tradeoff) for the approach where the server just reports  
what it's hearing.

Having had some useful exchanges on this topic, it also is becoming  
apparent to me that this event is poorly named, and we should  
consider renaming it to "INTERESTING-INPUT-HEARD" or something akin  
to that, because as others have pointed out, a DTMF-only recognizer  
will never detect "start of speech".

Another consideration to fold into the design choice is  
extensibility. Bear with me through a little gedankenexperiment.

Suppose we want to define a new recognizer type, which I'll call the  
"name that tune" recognizer. The client plays music to the server and  
the server recognizes musical notes. The grammar is a standard  
musical notation, augmented with a semantic interpretation that  
transforms the notes into the title of the tune and provides that as  
an answer.

First, there's no speech involved (or is there...hang on a minute).  
Second, in order to accommodate the "name that tune" recognizer, we'd  
have to extend both the client and the server to undetstand a  
directive as to whether to recognize music or now, inaddition to what  
the server already knows what to do based on the grammar. If you  
follow my logic above, whether or not we do that, we have to extend  
"start-of-speech" to say "I'm hearing music". So far fairly  
straightforward, but let me now throw in the pathological twist.

Suppose what I feed to a  combined music/speech recognizer is a work  
in sprechstimme (spoken music), like the "Geographical Fugue" (aside:  
this is a wonderful piece of music I highly recommend to anyone  
interested in small ensemble singing). In this case, the tune could  
be named by either doing speech or music recognition. Why is there  
any need for the client to constrain the server as to which it tries  
to do when it's already told the server what it wants through the  
grammar?

A few other comments below

> ii. Flexibility: Want speech-only recognition (because a second  
> recogniser is doing hotword on DTMF)
>
I don't see how flexibility is affected by this deisgn choice. If  
that's what you want, feed the speech-only recognizer a grammar  
without any DTMF rules.

> Option a is David's "do what I mean case"; option b is the extra  
> dial for the client.
>
Actually, that's not the point I was making with "do what I mean",  
but it's not essential to the discussion so let's move on.

> It is worth noting that VoiceXML uses option b. This allows one to  
> activate both speech grammars and DTMF grammars (and therefore be  
> informed of any errors in the grammars at activation time) but  
> independently turn on whichever input mode you like e.g. perhaps  
> start with "both" then change to "dtmf".
>
I'm not sure the VXML precedent is relevant here, because the  
application behind VXML is working a different part of the problem -  
how to traverse a TUI dialog based on different parts of the input  
space. In fact, I suspect that the VXML: choice was conditioned more  
by limitations at the time it was specified than an underlying good  
design choice. Clearly having to specify this in VXML make the job of  
handling a TUI with nodes like "Say or press 5" harder rather than  
easier.

Summing up, while I don't feel strongly one way or the other, I have  
a preference for handling this as follows:

a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or  
something equivalent.
b) Include a parameter in the event saying what was interesting about  
the input you received, with a registry of values which includes:
     - signal above noise floor
     - speech
     - dtmf
     - (possibly) music
c) allow the event to be generated multiple times during a request

Note that all of the above I'm saying with my technical hat on and my  
chair hat off.
Putting my chair hat on for a moment, we really need to get this spec  
to last call, so at some point Eric or I is going to declare rough  
consensus so we can move on.

Dave Oran.
> Dave
>
>
>
>
>
>
>
>
>
>
>
>
> Sarvi makes a good point that adding the reason why the START-OF- 
> SPEECH occurred does not fix the optimised bargin case.
>

---

From: "Dave Burke" <[email-masked]>
Date: Wed, 6 Jul 2005 15:29:12 +0100

Inline.

Dave

----- Original Message ----- 
From: "David R Oran" <[email-masked]>
To: "Dave Burke" <[email-masked]>
Cc: <[email-masked]>; "Shanmugham, Saravanan" <[email-masked]>; "Klaus 
Reifenrath" <[email-masked]>
Sent: Wednesday, July 06, 2005 1:07 PM
Subject: Re: [Speechsc] START-OF-SPEECH in DTMF-only mode -> summary(?)

>> bargin is different and less important to what input type(s)  the 

DB> Adding a parameter to the START-OF-SPEECH event would certainly allow 
the client to ignore (i.e. let prompts continue playing) the event if the 
event type is not of interest (e.g. the client would ignore speech start 
events when it is interested only in  DTMF start events). This _only_ works 
for the non-optimised case, however. For the optimised case, assuming 
START-OF-SPEECH coincides with the bargin signal to the speechsynth, prompts 
will stop playing for inputs that the client might not be interested in 
(e.g. a speech input will stop prompts playing even if the client is only 
interested in DTMF).

> Do you mean exactly this, or do you mean "for the client to indicate  to 
> the resource what input types it should look for"?

DB> Yes exactly - apologies for not being clear.

> I'm having difficulty following the logic of why noise would  necessarily 
> trigger START-OF-SPEECH if you were listening for speech  but not DTMF. I 
> suppose you can use a more forgiving discriminator if  all you need to 
> tell is if you're getting DTMF, but I've had a number  of real-world cases 
> where wind noise was detected as DTMF, and  there's always the ambiguity 
> when you have Captain Crunch on the  phone. In either case in the 
> non-optimized case it's the client who  gets to decide whether an event 
> should be interpreted as barge-in or  not, so it seems an aesthetic 
> protocol design decision whether the  client tells the server ahead of 
> time what circumstances to generate  the START-OF-SPEECH event for, or 
> whether the event gets generated  and the client decides based on what's 
> in the event whether it should  be treated as barge- in.
>
> I suppose one could make the argument that because the spec implies  that 
> the event can only be generated once per request that if a DTMF/ speech 
> capable recognizer first hears enough noise to think it's  hearing speech 
> and later hears DTMF, the client will declare barge-in  when the event 
> comes and not when he DTMF actually gets heard.
>
> If that's deemed a problem, we can still handle that in the design  where 
> the server just reports what it's hearing by allowing multiple  events to 
> be generated during a single request.
>
> client provides a filter for whether to generate the event or not, I  have 
> a mild preference (based on aesthetics rather than some hard  engineering 
> tradeoff) for the approach where the server just reports  what it's 
> hearing.
>
>
> apparent to me that this event is poorly named, and we should  consider 
> renaming it to "INTERESTING-INPUT-HEARD" or something akin  to that, 
> because as others have pointed out, a DTMF-only recognizer  will never 
> detect "start of speech".
>
> Another consideration to fold into the design choice is  extensibility. 
> Bear with me through a little gedankenexperiment.
>
> "name that tune" recognizer. The client plays music to the server and  the 
> server recognizes musical notes. The grammar is a standard  musical 
> notation, augmented with a semantic interpretation that  transforms the 
> notes into the title of the tune and provides that as  an answer.
>
> have to extend both the client and the server to undetstand a  directive 
> as to whether to recognize music or now, inaddition to what  the server 
> already knows what to do based on the grammar. If you  follow my logic 
> above, whether or not we do that, we have to extend  "start-of-speech" to 
> say "I'm hearing music". So far fairly  straightforward, but let me now 
> throw in the pathological twist.
>
> Suppose what I feed to a  combined music/speech recognizer is a work  in 
> sprechstimme (spoken music), like the "Geographical Fugue" (aside:  this 
> is a wonderful piece of music I highly recommend to anyone  interested in 
> small ensemble singing). In this case, the tune could  be named by either 
> doing speech or music recognition. Why is there  any need for the client 
> to constrain the server as to which it tries  to do when it's already told 
> the server what it wants through the  grammar?
>
> I don't see how flexibility is affected by this deisgn choice. If  that's 
> what you want, feed the speech-only recognizer a grammar  without any DTMF 
> rules.
>
>> + How to implement 5?
>>
>> a. Implicitly:
>> Option a is David's "do what I mean case"; option b is the extra  dial 
>> for the client.
>>
> Actually, that's not the point I was making with "do what I mean",  but 
> it's not essential to the discussion so let's move on.
>
>> independently turn on whichever input mode you like e.g. perhaps  start 
>> with "both" then change to "dtmf".
>>
> I'm not sure the VXML precedent is relevant here, because the  application 
> behind VXML is working a different part of the problem -  how to traverse 
> a TUI dialog based on different parts of the input  space. In fact, I 
> suspect that the VXML: choice was conditioned more  by limitations at the 
> time it was specified than an underlying good  design choice. Clearly 
> having to specify this in VXML make the job of  handling a TUI with nodes 
> like "Say or press 5" harder rather than  easier.

DB> The VoiceXML edge-case is pretty weird so it's not a major concern. My 
main concern is that the client can indicate, somehow, what the input modes 
are.

>
> Summing up, while I don't feel strongly one way or the other, I have  a 
> preference for handling this as follows:
>
> a) Rename "START-OF-SPEECH" to "INTERESTING-INPUT-RECEIVED" or  something 
> equivalent.
> b) Include a parameter in the event saying what was interesting about  the 
> input you received, with a registry of values which includes:

DB> I like these suggestions (START-OF-INPUT?). However, I don't see how the 
problem of the optimised case is not solved by them. I think the optimised 
case is fine if we have the following rules:

1. START-OF-SPEECH (and optimised bargin) is only generated for the input 
type that is being listened for
2. A speechrecog listens for DTMF if DTMF grammars are active, speech if 
speech grammars are active, or speech and DTMF if both grammar types are 
active.

> Putting my chair hat on for a moment, we really need to get this spec  to 
> last call, so at some point Eric or I is going to declare rough  consensus 
> so we can move on.

DB> Agreed!

>
> Dave Oran.
>> Dave
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Sarvi makes a good point that adding the reason why the START-OF- SPEECH

---

From: David R Oran <[email-masked]>
Date: Wed, 6 Jul 2005 10:45:06 -0400

I think we're getting close. I though about snipping out some pieces  
to cut down the text, but I realized the context is still needed. See  
inline.

On Jul 6, 2005, at 10:29 AM, Dave Burke wrote:

> Inline.
>
> Dave
>
> ----- Original Message ----- From: "David R Oran" <[email-masked]>
> Cc: <[email-masked]>; "Shanmugham, Saravanan" <[email-masked]>;  
> "Klaus Reifenrath" <[email-masked]>
>>> 2. START-OF-SPEECH is useful for the client to calculate the  
>>> bargin  time (e.g. VoiceXML 2.1 <mark>)
>>> 4. Because of the previous point, the question of what input  
>>> type  caused bargin is different and less important to what input  
>>> type(s)  the recogniser is listening for
>>>
> DB> Adding a parameter to the START-OF-SPEECH event would certainly  
> allow the client to ignore (i.e. let prompts continue playing) the  
> event if the event type is not of interest (e.g. the client would  
> ignore speech start events when it is interested only in  DTMF  
> start events). This _only_ works for the non-optimised case,  
> however. For the optimised case, assuming START-OF-SPEECH coincides  
> with the bargin signal to the speechsynth, prompts will stop  
> playing for inputs that the client might not be interested in (e.g.  
> a speech input will stop prompts playing even if the client is only  
> interested in DTMF).
>
OK, now I get it. There's a need for the client to both handle the  
non-optimized case itself, and influence or at least have a clue what  
the server is going to do in the optimized case.

>
>>
>>
>>> 5. Currently, MRCPv2 has no way of indicating what input type(s)  
>>> a recogniser is listening for
>>>
>>>
>> Do you mean exactly this, or do you mean "for the client to  
>> indicate  to the resource what input types it should look for"?
>>
>
>> necessarily trigger START-OF-SPEECH if you were listening for  
>> speech  but not DTMF. I suppose you can use a more forgiving  
>> discriminator if  all you need to tell is if you're getting DTMF,  
>> but I've had a number  of real-world cases where wind noise was  
>> detected as DTMF, and  there's always the ambiguity when you have  
>> Captain Crunch on the  phone. In either case in the non-optimized  
>> case it's the client who  gets to decide whether an event should  
>> be interpreted as barge-in or  not, so it seems an aesthetic  
>> protocol design decision whether the  client tells the server  
>> ahead of time what circumstances to generate  the START-OF-SPEECH  
>> event for, or whether the event gets generated  and the client  
>> decides based on what's in the event whether it should  be treated  
>> as barge- in.
>>
>> I suppose one could make the argument that because the spec  
>> implies  that the event can only be generated once per request  
>> that if a DTMF/ speech capable recognizer first hears enough noise  
>> to think it's  hearing speech and later hears DTMF, the client  
>> will declare barge-in  when the event comes and not when he DTMF  
>> actually gets heard.
>>
>> If that's deemed a problem, we can still handle that in the  
>> design  where the server just reports what it's hearing by  
>> allowing multiple  events to be generated during a single request.
>>
>> Between the approach just outlined above, and an approach where  
>> the client provides a filter for whether to generate the event or  
>> not, I  have a mild preference (based on aesthetics rather than  
>> some hard  engineering tradeoff) for the approach where the server  
>> just reports  what it's hearing.
>>
>>
>> Having had some useful exchanges on this topic, it also is  
>> becoming apparent to me that this event is poorly named, and we  
>> should  consider renaming it to "INTERESTING-INPUT-HEARD" or  
>> something akin  to that, because as others have pointed out, a  
>> DTMF-only recognizer  will never detect "start of speech".
>>
>> Suppose we want to define a new recognizer type, which I'll call  
>> the "name that tune" recognizer. The client plays music to the  
>> server and  the server recognizes musical notes. The grammar is a  
>> standard  musical notation, augmented with a semantic  
>> interpretation that  transforms the notes into the title of the  
>> tune and provides that as  an answer.
>>
>> First, there's no speech involved (or is there...hang on a  
>> minute). Second, in order to accommodate the "name that tune"  
>> recognizer, we'd have to extend both the client and the server to  
>> undetstand a  directive as to whether to recognize music or now,  
>> inaddition to what  the server already knows what to do based on  
>> the grammar. If you  follow my logic above, whether or not we do  
>> that, we have to extend  "start-of-speech" to say "I'm hearing  
>> music". So far fairly  straightforward, but let me now throw in  
>> the pathological twist.
>>
>> Suppose what I feed to a  combined music/speech recognizer is a  
>> work  in sprechstimme (spoken music), like the "Geographical  
>> Fugue" (aside:  this is a wonderful piece of music I highly  
>> recommend to anyone  interested in small ensemble singing). In  
>> this case, the tune could  be named by either doing speech or  
>> music recognition. Why is there  any need for the client to  
>> constrain the server as to which it tries  to do when it's already  
>> told the server what it wants through the  grammar?
>>
>> Actually, that's not the point I was making with "do what I  
>> mean",  but it's not essential to the discussion so let's move on.
>>
>>
>>> It is worth noting that VoiceXML uses option b. This allows one  
>>> to activate both speech grammars and DTMF grammars (and therefore  
>>> be informed of any errors in the grammars at activation time) but  
>> application behind VXML is working a different part of the problem  
>> -  how to traverse a TUI dialog based on different parts of the  
>> input  space. In fact, I suspect that the VXML: choice was  
>> conditioned more  by limitations at the time it was specified than  
>> an underlying good  design choice. Clearly having to specify this  
>> in VXML make the job of  handling a TUI with nodes like "Say or  
>> press 5" harder rather than  easier.
>>
>
> DB> The VoiceXML edge-case is pretty weird so it's not a major  
> concern. My main concern is that the client can indicate, somehow,  
> what the input modes are.
>
>
>>
>> Summing up, while I don't feel strongly one way or the other, I  
>> have  a preference for handling this as follows:
>>
>> b) Include a parameter in the event saying what was interesting  
>> about  the input you received, with a registry of values which  
>> includes:
> DB> I like these suggestions (START-OF-INPUT?). However, I don't  
> see how the problem of the optimised case is not solved by them. I  
> think the optimised case is fine if we have the following rules:
>
Yes, I hadn't thought through the optimized case as thoroughly as  
you. Your suggested method name is fine by me as well

> 1. START-OF-SPEECH (and optimised bargin) is only generated for the  
> input type that is being listened for
> 2. A speechrecog listens for DTMF if DTMF grammars are active,  
> speech if speech grammars are active, or speech and DTMF if both  
> grammar types are active.
>
Works for me.

>
>> Note that all of the above I'm saying with my technical hat on and  
>> my chair hat off.
>> Putting my chair hat on for a moment, we really need to get this  
>> spec  to last call, so at some point Eric or I is going to declare  
>> rough  consensus so we can move on.
>>
>
> DB> Agreed!
>
>
>>
>> Dave Oran.
>>
>>> Dave
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>