Subject: RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA: ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

From: "Eric Burger" <[email-masked]>
Date: Tue, 18 Oct 2005 22:48:48 -0400

Fine with me, so long as server can reject and client can handle situation where I ask for only NLSML in SIP negotiation and then I ask for EMMA and the server doesn't handle it.

 -----Original Message-----
From: 	Shanmugham, Saravanan [mailto:[email-masked]]
Sent:	Tue Oct 18 20:57:20 2005
To:	Eric Burger; Brett Gavagni
Cc:	IETF SPEECHSC (E-mail)
Subject:	RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA: ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

I suggest that we do both. 
The SIP Alow/Accept header during MRCP session setup for server
discovery and routing.
We then add support for the Accept header in MRCPv2 messages. I suspect
this will be initially only used for RECOGNIZE/SET-PARAMS/GET-PARAMS.
And can be used in the case where the client or the server can do both.

What do you think.

Sarvi 

     -----Original Message-----
     From: Eric Burger [mailto:[email-masked]] 
     Sent: Monday, October 17, 2005 1:52 PM
     To: Shanmugham, Saravanan; Brett Gavagni
     Cc: IETF SPEECHSC (E-mail)
     Subject: Question on negotiating EMMA (was RE: [Speechsc] 
     EMMA: ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     
     My logic, which may be flawed, is that we want to be able 
     to achieve these objectives.
       o  Ensure the trivial ability to negotiate other result
          formats, such as EMMA.
       o  Ensure the server knows what format the client wants.
       o  Ensure the client knows the server supports the format
          the client wants, before it is "too late".
     
     I would offer that using SET-PARAMS is "too late", in that 
     the client and server has already spent the effort to 
     establish the RTP sessions.
     It would be a pity to then tear it all down when the 
     client finds out the server does not support the format of choice.
     
     One can negotiating the format at session establishment 
     time with SDP
     (a=resultformat:application/emma-xml) or with SIP 
     (Allow/Accept).  I would offer that it is easier to 
     implement, and makes MRCPv2 (SIP) proxies easier if we do 
     the negotiation at the SIP level.  For example, if a 
     client wants results in EMMA, a MRCPv2 proxy can route the 
     request to a server that supports EMMA by inspecting the 
     SIP headers, rather than having to dive in to the SDP.
     
     While the preceding paragraph indicates my preference for 
     using the SIP negotiation mechanism, there still is the 
     case where both the client and server support NLSML and EMMA.
     
     ++++ THE QUESTION ++++
     Is there a realistic use case where the client, in any 
     given session, would want to use multiple result formats?  
     That is, something like, "RECOGNIZE this and give me the 
     result in EMMA" and then later issue something like, 
     "RECOGNIZE that and give me the result in NLSML"?
     
     If so, then we would *also* need a per-recognition / 
     SET-PARAMS header for the result type.
     
     -----Original Message-----
     From: [email-masked] 
     [mailto:[email-masked]] On Behalf Of 
     Shanmugham, Saravanan
     Sent: Wednesday, September 28, 2005 2:36 PM
     To: Brett Gavagni; Jerry Carter
     Cc: IETF SPEECHSC (E-mail); [email-masked]; Baggia Paolo
     Subject: RE: [Speechsc] EMMA: 
     ExtensibleMultiModalAnnotation markuplanguagesupport?
     
     Would something like the HTTP "Accept" header address 
     these and other similar concerns.
     The RECOGNIZE request can then carry this header in it to 
     specify an alternative to NLSML.
     
     Sarvi 
     
          -----Original Message-----
          From: [email-masked] 
          [mailto:[email-masked]] On Behalf Of Brett Gavagni
          Sent: Saturday, September 24, 2005 7:51 AM
          To: Jerry Carter
          Cc: IETF SPEECHSC (E-mail); 
     [email-masked]; Baggia Paolo
          Subject: Re: [Speechsc] EMMA: Extensible 
          MultiModalAnnotation markuplanguagesupport?
          
          Hi,
          
          I agree that the ability for a client to request the 
          format of the recognizer result data content would be useful.
          
          I would prefer to see this function defined in a 
          consistent manner with the current draft specification; by 
          a client request to enable a session parameter via a 
          header (ie. "Result-Data-Type") in a SET-PARAMS or 
          RECOGNIZE request, rather than an attribute used for 
          RECOGNIZER session negotiation/update. The server should 
          then respond with an accept or reject response as it does 
          today for other client requested parameter values. 
          Currently, the draft specification doesn't expose the MRCP 
          resource configurable parameters in the SDP.
          
          I don't think that enabling platform-specific formats 
          facilitates the adoption of a standard specification for 
          speech resources. I  believe that the specification should 
          continue to address the required supported formats and 
          require new draft iterations including the updated 
          specifications. The updated draft iterations would 
          continue to assist with potential client/server 
          interoperability issues.
          
          Thanks,
          
          Brett Gavagni
          WebSphere Voice Server Development
          http://www-306.ibm.com/software/pervasive/voice_server/
          [email-masked]

          
          Jerry Carter <[email-masked]>
          Sent by: [email-masked]
          09/23/2005 11:42 PM
          
          To
          Baggia Paolo <[email-masked]> cc "IETF SPEECHSC 
          \(E-mail\)" <[email-masked]> Subject
          Re: [Speechsc] EMMA: Extensible MultiModal Annotation 
          markuplanguagesupport?

          
          I agree that minor changes made today will prevent 
     the need for an 
          amended document in the very near future.  Providing 
     for content 
          negotiation solves both the EMMA issue and allows for 
     future or 
          platform-specific return formats without requiring that the 
          specification be iterated.
          
          Of the two suggestions, I prefer the SDP extension as the 
          return format 
          is analogous to other SIP media type negotiations.

          On Sep 23, 2005, at 4:55 AM, Baggia Paolo wrote:
          > the last answers and discussions in this thread 
     seems to confirm
          > that the direction of MRCPv2 is to allow the use of 
     EMMA when
          > it will become a W3C Recommendation. Some text 
     should be added
          > in the future release to say that.
          > This is fine from us point of view, but a mechanism 
     for asking
          > a different result format is not present in the 
     current draft.
          >
          > We think to add that mechanism will allow a smooth 
     transition
          > from the current situation: always result in 
          "application/nlsml+xml",
          > to a future one with results in EMMA.
          >
          > Would it be possible to consider this "negotiation" 
     of the output
          > result in the next draft?
          >
          > There are different options to implement it:
          > a) to extend SDP to leave taht outside of MRCPv2 
     core protocol
          > E.g.
          > a=recogResultFormat:application/nlsml+xml
          > if present it sets the recognition result for a whole MRCP
          > session.
          >
          > b) to add a new recognizer header to set the result format
          >    for that specific recognition turn.
          >
          > What do you think?
          >
          > We are aware the draft is near to be closed, but we 
     think this
          > will help the passage from NLSML to EMMA.
          >
          > Regards,
          > Paolo Baggia, Vittorio Manzone, Patrizio Bergallo (Loquendo)

          _______________________________________________
          Speechsc mailing list
          [email-masked]
          https://www1.ietf.org/mailman/listinfo/speechsc

          _______________________________________________
          Speechsc mailing list
          [email-masked]
          https://www1.ietf.org/mailman/listinfo/speechsc

     _______________________________________________
     Speechsc mailing list
     [email-masked]
     https://www1.ietf.org/mailman/listinfo/speechsc
     
_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: Brett Gavagni <[email-masked]>
Date: Wed, 19 Oct 2005 06:31:53 -0400

I not too thrilled with the implication of leaking more MRCP Application 
parameters into SDP,  which could be possibly considered outside the scope 
of resource session establishment,. This could potentially end up 
complicating interoperability scenarios.

Thanks,

Brett Gavagni 

"Eric Burger" <[email-masked]> 
10/18/2005 10:48 PM

To
"Shanmugham, Saravanan" <[email-masked]>, Brett Gavagni/West Palm 
Beach/IBM@IBMUS
cc
"IETF SPEECHSC \(E-mail\)" <[email-masked]>
Subject
RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA: 
ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

Fine with me, so long as server can reject and client can handle situation 
where I ask for only NLSML in SIP negotiation and then I ask for EMMA and 
the server doesn't handle it.

 -----Original Message-----
From:   Shanmugham, Saravanan [mailto:[email-masked]]
Sent:   Tue Oct 18 20:57:20 2005
To:     Eric Burger; Brett Gavagni
Cc:     IETF SPEECHSC (E-mail)
Subject:        RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA: 
ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

What do you think.

Sarvi

     -----Original Message-----
     From: Eric Burger [mailto:[email-masked]]
     Sent: Monday, October 17, 2005 1:52 PM
     To: Shanmugham, Saravanan; Brett Gavagni
     Cc: IETF SPEECHSC (E-mail)
     Subject: Question on negotiating EMMA (was RE: [Speechsc]
     EMMA: ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

 

     ++++ THE QUESTION ++++

     -----Original Message-----
     From: [email-masked]
     [mailto:[email-masked]] On Behalf Of
     Shanmugham, Saravanan
     Sent: Wednesday, September 28, 2005 2:36 PM
     To: Brett Gavagni; Jerry Carter
     Cc: IETF SPEECHSC (E-mail); [email-masked]; Baggia Paolo
     Subject: RE: [Speechsc] EMMA:
     ExtensibleMultiModalAnnotation markuplanguagesupport?

     Sarvi
 
          -----Original Message-----
 
          Hi,

 
 
          Thanks,
 
          Brett Gavagni

 
 
          To

 

     seems to confirm
     EMMA when
     should be added
     for asking
     current draft.
     transition
          "application/nlsml+xml",
     of the output
     core protocol
     think this

---

From: "Dave Burke" <[email-masked]>
Date: Wed, 19 Oct 2005 13:50:23 +0100

I have to side with Brett here.

The reality is that there are plenty of things that the client may find out 
about the server "too late". Here are the ones that will undoubtedly happen 
in practice:
    - a given language not supported
    - a given semantic interpretation format within the SRGS <tag>s not 
supported
    - speech enrollment not supported

In practice, a client will know the vendor of the speech resource servers 
identified by a SIP URI and will know what to expect; network administrators 
will know not to load balance / failover a hetergeonous vendor set.

It seems much cleaner to use Accept on the RECOGNIZE request so the server 
knows the format(s) to use for the RECOGNITION-COMPLETE final response. 
SET-OPTIONs/GET-OPTIONs can be used for stickyness.

Dave

----- Original Message ----- 
From: "Brett Gavagni" <[email-masked]>
To: "Eric Burger" <[email-masked]>
Cc: "IETF SPEECHSC (E-mail)" <[email-masked]>; "Shanmugham, Saravanan" 
<[email-masked]>
Sent: Wednesday, October 19, 2005 11:31 AM
Subject: RE: Question on negotiating EMMA (was RE: [Speechsc] 
EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

---

From: Andrew Wahbe <[email-masked]>
Date: Wed, 19 Oct 2005 10:57:28 -0400

The question is whether you want the client to know which homogeneous 
set to use, or if you want a proxy to do it for you. If you want a proxy 
to be able to do this for you then you want it in the SDP (or perhaps a 
SIP header but I'd prefer SDP) wouldn't you?

Also note that things like the required language(s) can change over the 
course of one call. You could update your SDP and ReInvite in this case 
(and the proxy would find an appropriate server) or (if it is client 
driven) you could switch over to another homogeneous set.

I thought that advertising your needs and capabilities in session 
establishment so that you could be connected to a  compatible server was 
a big part of the point in using SIP here. If we resort to saying that 
"network administrators should know how to load balance this properly", 
can't we also say that "network administrators should configure 
everything to use the same codec" etc?

Having said that, I think that one could still argue that the emma vs 
nlsml support is a stretch for session establishment... its some of the 
other examples you gave that I have issue with. I hope we don't rule 
these out. I haven't brought this up earlier because I seem to remember 
someone saying that this sort of thing could be the subject of another 
separate RFC (SDP to describe MRCP resource capabilities).

Andrew

Dave Burke wrote:

> The reality is that there are plenty of things that the client may 
> find out about the server "too late". Here are the ones that will 
> undoubtedly happen in practice:
> In practice, a client will know the vendor of the speech resource 
> servers identified by a SIP URI and will know what to expect; network 
> administrators will know not to load balance / failover a hetergeonous 
> vendor set.
>
> It seems much cleaner to use Accept on the RECOGNIZE request so the 
> server knows the format(s) to use for the RECOGNITION-COMPLETE final 
> response. SET-OPTIONs/GET-OPTIONs can be used for stickyness.
>
> Dave
>
> ----- Original Message ----- From: "Brett Gavagni" <[email-masked]>
> Cc: "IETF SPEECHSC (E-mail)" <[email-masked]>; "Shanmugham, 
> Saravanan" <[email-masked]>
>> EMMA:

---

From: Andrew Wahbe <[email-masked]>
Date: Wed, 19 Oct 2005 11:18:21 -0400

On reading my own response I realized that a re-invite wouldn't work 
with a proxy. oops. (please hold back your comments ;-). I don't think 
this changes my point though.

Andrew

Andrew Wahbe wrote:

> set to use, or if you want a proxy to do it for you. If you want a 
> proxy to be able to do this for you then you want it in the SDP (or 
> perhaps a SIP header but I'd prefer SDP) wouldn't you?
>
> Also note that things like the required language(s) can change over 
> the course of one call. You could update your SDP and ReInvite in this 
> case (and the proxy would find an appropriate server) or (if it is 
> client driven) you could switch over to another homogeneous set.
>
> establishment so that you could be connected to a  compatible server 
> was a big part of the point in using SIP here. If we resort to saying 
> that "network administrators should know how to load balance this 
> properly", can't we also say that "network administrators should 
> configure everything to use the same codec" etc?
>
> nlsml support is a stretch for session establishment... its some of 
> the other examples you gave that I have issue with. I hope we don't 
> rule these out. I haven't brought this up earlier because I seem to 
> remember someone saying that this sort of thing could be the subject 
> of another separate RFC (SDP to describe MRCP resource capabilities).
>
> Andrew
>
> Dave Burke wrote:
>
>>    - a given semantic interpretation format within the SRGS <tag>s 
>> not supported
>> administrators will know not to load balance / failover a 
>> hetergeonous vendor set.
>>

---

From: "Dave Burke" <[email-masked]>
Date: Wed, 19 Oct 2005 18:12:22 +0100

There is certainly a general question of how an MRCP server advertises 
capabilities and how the client requests particular capabilities. I do, 
however, believe that MRCP's choice of SIP for discovery and rendevous will 
still prove wise.  The mechanism for capability advertisement and 
specification might be best delegated to a separate draft at this point in 
MRCP's development. For now we've got DNS SRV  (a given AOR can indicate a 
particular capability set e.g. vendorx.en.example.com).

As far as mechanisms are considered, I'm still not keen on using SDP. A 
"pure" SIP proxy (as defined in RFC 3261) will not work by analysing SDP. 
Nor will registering an MRCP server capability set (since REGISTER does not 
contain SDP and it's Accept / Allow headers relate to the response to the 
REGISTER). That leaves headers: one framework that could be considered is 
based on RFC 3840/3841. For example, a speechrecog does a register:

REGISTER sip:example.com
To: [email-masked]
Contact: <sip:10.0.0.1>;language="en";reco-result-format="EMMA"

An MRCP client looking for an ASR engine supporting EMMA and English places 
the following request to a proxy.

INVITE sip:[email-masked]
Accept-Contact: *;reco-result-format="EMMA";language="en"

The OPTIONS method (already mentioned in MRCPv2) can also be used (it 
returns the Contact header with the feature parameters).

Dave

----- Original Message ----- 
From: "Andrew Wahbe" <[email-masked]>
To: "Andrew Wahbe" <[email-masked]>
Cc: "Dave Burke" <[email-masked]>; "IETF SPEECHSC (E-mail)" 
<[email-masked]>; "Shanmugham, Saravanan" <[email-masked]>; "Eric 
Burger" <[email-masked]>
Sent: Wednesday, October 19, 2005 4:18 PM
Subject: Re: Question on negotiating EMMA (was RE: [Speechsc] 
EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)