Subject: RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

From: "Shanmugham, Saravanan" <[email-masked]>
Date: Wed, 19 Oct 2005 10:30:53 -0700

Discovery for MRCPv2 capabilities in general is through SIP options as
described in Section 7 of the draft. 
 
On the topic of EMMA/NLSML.

After reading some of the emails. I am rethinking my previous acceptance
of Eric's SIP Accept idea.

I think his alternate suggestion of making it part of the SDP for the
session makes more sense for the following reasons.

   1. It works well with the above discovery mechanism which describe
capabilities through SDP.
   2. It works for routing as such a proxy can take look at the SDP to
make this decision. 
   3. A routing proxy would any way have to look at the SDP for other
things such as resource type.
   4. It also allows this to be negotiated during Session Setup through
the INVITE offer answer model.

I would also suggest that we still add the Accept Header similar to HTTP
into MRCP messages to be used in messages like, SET-PARAMS, GET-PARAMS,
RECOGNIZE, INTERPRET as well. This allows us to change format for
individual requests or the session as necessry if the resource supports
more than one format.

Sarvi

     -----Original Message-----
     From: Dave Burke [mailto:[email-masked]] 
     Sent: Wednesday, October 19, 2005 10:12 AM
     To: Andrew Wahbe
     Cc: IETF SPEECHSC (E-mail); Shanmugham, Saravanan; Eric Burger
     Subject: Re: Question on negotiating EMMA (was RE: 
     [Speechsc] 
     EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     
     There is certainly a general question of how an MRCP 
     server advertises capabilities and how the client requests 
     particular capabilities. I do, however, believe that 
     MRCP's choice of SIP for discovery and rendevous will 
     still prove wise.  The mechanism for capability 
     advertisement and specification might be best delegated to 
     a separate draft at this point in MRCP's development. For 
     now we've got DNS SRV  (a given AOR can indicate a 
     particular capability set e.g. vendorx.en.example.com).
     
     As far as mechanisms are considered, I'm still not keen on 
     using SDP. A "pure" SIP proxy (as defined in RFC 3261) 
     will not work by analysing SDP. 
     Nor will registering an MRCP server capability set (since 
     REGISTER does not contain SDP and it's Accept / Allow 
     headers relate to the response to the REGISTER). That 
     leaves headers: one framework that could be considered is 
     based on RFC 3840/3841. For example, a speechrecog does a register:
     
     REGISTER sip:example.com
     To: [email-masked]
     Contact: <sip:10.0.0.1>;language="en";reco-result-format="EMMA"
     
     An MRCP client looking for an ASR engine supporting EMMA 
     and English places the following request to a proxy.
     
     INVITE sip:[email-masked]
     Accept-Contact: *;reco-result-format="EMMA";language="en"
     
     The OPTIONS method (already mentioned in MRCPv2) can also 
     be used (it returns the Contact header with the feature 
     parameters).
     
     Dave
     
     ----- Original Message -----
     From: "Andrew Wahbe" <[email-masked]>
     To: "Andrew Wahbe" <[email-masked]>
     Cc: "Dave Burke" <[email-masked]>; "IETF 
     SPEECHSC (E-mail)" 
     <[email-masked]>; "Shanmugham, Saravanan" 
     <[email-masked]>; "Eric Burger" <[email-masked]>
     Sent: Wednesday, October 19, 2005 4:18 PM
     Subject: Re: Question on negotiating EMMA (was RE: [Speechsc]
     EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)

     > On reading my own response I realized that a re-invite 
     wouldn't work 
     > with a proxy. oops. (please hold back your comments ;-). 
     I don't think 
     > this changes my point though.
     >
     > Andrew
     >
     > Andrew Wahbe wrote:
     >
     >> The question is whether you want the client to know 
     which homogeneous 
     >> set to use, or if you want a proxy to do it for you. If 
     you want a 
     >> proxy to be able to do this for you then you want it in 
     the SDP (or 
     >> perhaps a SIP header but I'd prefer SDP) wouldn't you?
     >>
     >> Also note that things like the required language(s) can 
     change over 
     >> the course of one call. You could update your SDP and 
     ReInvite in 
     >> this case (and the proxy would find an appropriate 
     server) or (if it 
     >> is client driven) you could switch over to another 
     homogeneous set.
     >>
     >> I thought that advertising your needs and capabilities 
     in session 
     >> establishment so that you could be connected to a  
     compatible server 
     >> was a big part of the point in using SIP here. If we 
     resort to saying 
     >> that "network administrators should know how to load 
     balance this 
     >> properly", can't we also say that "network 
     administrators should 
     >> configure everything to use the same codec" etc?
     >>
     >> Having said that, I think that one could still argue 
     that the emma vs 
     >> nlsml support is a stretch for session establishment... 
     its some of 
     >> the other examples you gave that I have issue with. I 
     hope we don't 
     >> rule these out. I haven't brought this up earlier 
     because I seem to 
     >> remember someone saying that this sort of thing could 
     be the subject 
     >> of another separate RFC (SDP to describe MRCP resource 
     capabilities).
     >>
     >> Andrew
     >>
     >> Dave Burke wrote:
     >>
     >>> I have to side with Brett here.
     >>>
     >>> The reality is that there are plenty of things that 
     the client may 
     >>> find out about the server "too late". Here are the 
     ones that will 
     >>> undoubtedly happen in practice:
     >>>    - a given language not supported
     >>>    - a given semantic interpretation format within the 
     SRGS <tag>s 
     >>> not supported
     >>>    - speech enrollment not supported
     >>>
     >>> In practice, a client will know the vendor of the 
     speech resource 
     >>> servers identified by a SIP URI and will know what to expect; 
     >>> network administrators will know not to load balance / 
     failover a 
     >>> hetergeonous vendor set.
     >>>
     >>> It seems much cleaner to use Accept on the RECOGNIZE 
     request so the 
     >>> server knows the format(s) to use for the 
     RECOGNITION-COMPLETE final 
     >>> response. SET-OPTIONs/GET-OPTIONs can be used for stickyness.
     >>>
     >>> Dave
     >>>
     >>> ----- Original Message ----- From: "Brett Gavagni" 
     >>> <[email-masked]>
     >>> To: "Eric Burger" <[email-masked]>
     >>> Cc: "IETF SPEECHSC (E-mail)" <[email-masked]>; "Shanmugham, 
     >>> Saravanan" <[email-masked]>
     >>> Sent: Wednesday, October 19, 2005 11:31 AM
     >>> Subject: RE: Question on negotiating EMMA (was RE: [Speechsc]
     >>> EMMA:ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     >>>
     >>>
     >>>> I not too thrilled with the implication of leaking more MRCP 
     >>>> Application parameters into SDP,  which could be possibly 
     >>>> considered outside the scope of resource session 
     establishment,. 
     >>>> This could potentially end up complicating interoperability 
     >>>> scenarios.
     >>>>
     >>>> Thanks,
     >>>>
     >>>> Brett Gavagni
     >>>> WebSphere Voice Server Development
     >>>> http://www-306.ibm.com/software/pervasive/voice_server/
     >>>> [email-masked]
     >>>>
     >>>>
     >>>>
     >>>>
     >>>> "Eric Burger" <[email-masked]>
     >>>> 10/18/2005 10:48 PM
     >>>>
     >>>> To
     >>>> "Shanmugham, Saravanan" <[email-masked]>, Brett 
     Gavagni/West Palm 
     >>>> Beach/IBM@IBMUS cc "IETF SPEECHSC \(E-mail\)" 
     <[email-masked]> 
     >>>> Subject
     >>>> RE: Question on negotiating EMMA (was RE: [Speechsc] EMMA:
     >>>> ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>
     >>>> Fine with me, so long as server can reject and client 
     can handle 
     >>>> situation where I ask for only NLSML in SIP 
     negotiation and then I 
     >>>> ask for EMMA and the server doesn't handle it.
     >>>>
     >>>> -----Original Message-----
     >>>> From:   Shanmugham, Saravanan [mailto:[email-masked]]
     >>>> Sent:   Tue Oct 18 20:57:20 2005
     >>>> To:     Eric Burger; Brett Gavagni
     >>>> Cc:     IETF SPEECHSC (E-mail)
     >>>> Subject:        RE: Question on negotiating EMMA (was 
     RE: [Speechsc]
     >>>> EMMA:
     >>>> ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     >>>>
     >>>> I suggest that we do both.
     >>>> The SIP Alow/Accept header during MRCP session setup 
     for server 
     >>>> discovery and routing.
     >>>> We then add support for the Accept header in MRCPv2 
     messages. I 
     >>>> suspect this will be initially only used for 
     RECOGNIZE/SET-PARAMS/GET-PARAMS.
     >>>> And can be used in the case where the client or the 
     server can do both.
     >>>>
     >>>> What do you think.
     >>>>
     >>>> Sarvi
     >>>>
     >>>>     -----Original Message-----
     >>>>     From: Eric Burger [mailto:[email-masked]]
     >>>>     Sent: Monday, October 17, 2005 1:52 PM
     >>>>     To: Shanmugham, Saravanan; Brett Gavagni
     >>>>     Cc: IETF SPEECHSC (E-mail)
     >>>>     Subject: Question on negotiating EMMA (was RE: [Speechsc]
     >>>>     EMMA: 
     ExtensibleMultiModalAnnotationmarkuplanguagesupport?)
     >>>>
     >>>>     My logic, which may be flawed, is that we want to be able
     >>>>     to achieve these objectives.
     >>>>       o  Ensure the trivial ability to negotiate other result
     >>>>          formats, such as EMMA.
     >>>>       o  Ensure the server knows what format the client wants.
     >>>>       o  Ensure the client knows the server supports 
     the format
     >>>>          the client wants, before it is "too late".
     >>>>
     >>>>     I would offer that using SET-PARAMS is "too late", in that
     >>>>     the client and server has already spent the effort to
     >>>>     establish the RTP sessions.
     >>>>     It would be a pity to then tear it all down when the
     >>>>     client finds out the server does not support the 
     format of choice.
     >>>>
     >>>>     One can negotiating the format at session establishment
     >>>>     time with SDP
     >>>>     (a=resultformat:application/emma-xml) or with SIP
     >>>>     (Allow/Accept).  I would offer that it is easier to
     >>>>     implement, and makes MRCPv2 (SIP) proxies easier if we do
     >>>>     the negotiation at the SIP level.  For example, if a
     >>>>     client wants results in EMMA, a MRCPv2 proxy can route the
     >>>>     request to a server that supports EMMA by inspecting the
     >>>>     SIP headers, rather than having to dive in to the SDP.
     >>>>
     >>>>     While the preceding paragraph indicates my preference for
     >>>>     using the SIP negotiation mechanism, there still is the
     >>>>     case where both the client and server support 
     NLSML and EMMA.
     >>>>
     >>>>     ++++ THE QUESTION ++++
     >>>>     Is there a realistic use case where the client, in any
     >>>>     given session, would want to use multiple result formats?
     >>>>     That is, something like, "RECOGNIZE this and give me the
     >>>>     result in EMMA" and then later issue something like,
     >>>>     "RECOGNIZE that and give me the result in NLSML"?
     >>>>
     >>>>     If so, then we would *also* need a per-recognition /
     >>>>     SET-PARAMS header for the result type.
     >>>>
     >>>>     -----Original Message-----
     >>>>     From: [email-masked]
     >>>>     [mailto:[email-masked]] On Behalf Of
     >>>>     Shanmugham, Saravanan
     >>>>     Sent: Wednesday, September 28, 2005 2:36 PM
     >>>>     To: Brett Gavagni; Jerry Carter
     >>>>     Cc: IETF SPEECHSC (E-mail); 
     [email-masked]; Baggia Paolo
     >>>>     Subject: RE: [Speechsc] EMMA:
     >>>>     ExtensibleMultiModalAnnotation markuplanguagesupport?
     >>>>
     >>>>     Would something like the HTTP "Accept" header address
     >>>>     these and other similar concerns.
     >>>>     The RECOGNIZE request can then carry this header in it to
     >>>>     specify an alternative to NLSML.
     >>>>
     >>>>     Sarvi
     >>>>
     >>>>          -----Original Message-----
     >>>>          From: [email-masked]
     >>>>          [mailto:[email-masked]] On Behalf 
     Of Brett Gavagni
     >>>>          Sent: Saturday, September 24, 2005 7:51 AM
     >>>>          To: Jerry Carter
     >>>>          Cc: IETF SPEECHSC (E-mail);
     >>>>     [email-masked]; Baggia Paolo
     >>>>          Subject: Re: [Speechsc] EMMA: Extensible
     >>>>          MultiModalAnnotation markuplanguagesupport?
     >>>>
     >>>>          Hi,
     >>>>
     >>>>          I agree that the ability for a client to request the
     >>>>          format of the recognizer result data content 
     would be useful.
     >>>>
     >>>>          I would prefer to see this function defined in a
     >>>>          consistent manner with the current draft 
     specification; by
     >>>>          a client request to enable a session parameter via a
     >>>>          header (ie. "Result-Data-Type") in a SET-PARAMS or
     >>>>          RECOGNIZE request, rather than an attribute used for
     >>>>          RECOGNIZER session negotiation/update. The 
     server should
     >>>>          then respond with an accept or reject 
     response as it does
     >>>>          today for other client requested parameter values.
     >>>>          Currently, the draft specification doesn't 
     expose the MRCP
     >>>>          resource configurable parameters in the SDP.
     >>>>
     >>>>          I don't think that enabling platform-specific formats
     >>>>          facilitates the adoption of a standard 
     specification for
     >>>>          speech resources. I  believe that the 
     specification should
     >>>>          continue to address the required supported 
     formats and
     >>>>          require new draft iterations including the updated
     >>>>          specifications. The updated draft iterations would
     >>>>          continue to assist with potential client/server
     >>>>          interoperability issues.
     >>>>
     >>>>          Thanks,
     >>>>
     >>>>          Brett Gavagni
     >>>>          WebSphere Voice Server Development
     >>>>          
     http://www-306.ibm.com/software/pervasive/voice_server/
     >>>>          [email-masked]
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>          Jerry Carter <[email-masked]>
     >>>>          Sent by: [email-masked]
     >>>>          09/23/2005 11:42 PM
     >>>>
     >>>>          To
     >>>>          Baggia Paolo <[email-masked]> cc 
     "IETF SPEECHSC
     >>>>          \(E-mail\)" <[email-masked]> Subject
     >>>>          Re: [Speechsc] EMMA: Extensible MultiModal Annotation
     >>>>          markuplanguagesupport?
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>
     >>>>          I agree that minor changes made today will prevent
     >>>>     the need for an
     >>>>          amended document in the very near future.  Providing
     >>>>     for content
     >>>>          negotiation solves both the EMMA issue and allows for
     >>>>     future or
     >>>>          platform-specific return formats without 
     requiring that the
     >>>>          specification be iterated.
     >>>>
     >>>>          Of the two suggestions, I prefer the SDP 
     extension as the
     >>>>          return format
     >>>>          is analogous to other SIP media type negotiations.
     >>>>
     >>>>
     >>>>          On Sep 23, 2005, at 4:55 AM, Baggia Paolo wrote:
     >>>>          > the last answers and discussions in this thread
     >>>>     seems to confirm
     >>>>          > that the direction of MRCPv2 is to allow the use of
     >>>>     EMMA when
     >>>>          > it will become a W3C Recommendation. Some text
     >>>>     should be added
     >>>>          > in the future release to say that.
     >>>>          > This is fine from us point of view, but a mechanism
     >>>>     for asking
     >>>>          > a different result format is not present in the
     >>>>     current draft.
     >>>>          >
     >>>>          > We think to add that mechanism will allow a smooth
     >>>>     transition
     >>>>          > from the current situation: always result in
     >>>>          "application/nlsml+xml",
     >>>>          > to a future one with results in EMMA.
     >>>>          >
     >>>>          > Would it be possible to consider this "negotiation"
     >>>>     of the output
     >>>>          > result in the next draft?
     >>>>          >
     >>>>          > There are different options to implement it:
     >>>>          > a) to extend SDP to leave taht outside of MRCPv2
     >>>>     core protocol
     >>>>          > E.g.
     >>>>          > a=recogResultFormat:application/nlsml+xml
     >>>>          > if present it sets the recognition result 
     for a whole MRCP
     >>>>          > session.
     >>>>          >
     >>>>          > b) to add a new recognizer header to set 
     the result format
     >>>>          >    for that specific recognition turn.
     >>>>          >
     >>>>          > What do you think?
     >>>>          >
     >>>>          > We are aware the draft is near to be closed, but we
     >>>>     think this
     >>>>          > will help the passage from NLSML to EMMA.
     >>>>          >
     >>>>          > Regards,
     >>>>          > Paolo Baggia, Vittorio Manzone, Patrizio Bergallo 
     >>>> (Loquendo)
     >>>>
     >>>>
     >>>>          _______________________________________________
     >>>>          Speechsc mailing list
     >>>>          [email-masked]
     >>>>          https://www1.ietf.org/mailman/listinfo/speechsc
     >>>>
     >>>>
     >>>>
     >>>>          _______________________________________________
     >>>>          Speechsc mailing list
     >>>>          [email-masked]
     >>>>          https://www1.ietf.org/mailman/listinfo/speechsc
     >>>>
     >>>>
     >>>>     _______________________________________________
     >>>>     Speechsc mailing list
     >>>>     [email-masked]
     >>>>     https://www1.ietf.org/mailman/listinfo/speechsc
     >>>>
     >>>>
     >>>>
     >>>> _______________________________________________
     >>>> Speechsc mailing list
     >>>> [email-masked]
     >>>> https://www1.ietf.org/mailman/listinfo/speechsc
     >>>>
     >>>
     >>>
     >>> _______________________________________________
     >>> Speechsc mailing list
     >>> [email-masked]
     >>> https://www1.ietf.org/mailman/listinfo/speechsc
     >>>
     >>>
     >>_______________________________________________
     >>Speechsc mailing list
     >>[email-masked]
     >>https://www1.ietf.org/mailman/listinfo/speechsc
     >>
     >>
     > 

_______________________________________________
Speechsc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speechsc

---

From: David R Oran <[email-masked]>
Date: Wed, 19 Oct 2005 15:31:40 -0400

>    2. It works for routing as such a proxy can take look at the SDP to
Really? What if it's SMIME encrypted. i would council against putting  
anythng needed for deciding which server to contact or how to route  
the request into the SDP.

> things such as resource type.
If so, we've probably made a design error and should have it in SIP  
caller prefs instead.

> the INVITE offer answer model.
That is true.