text:
|Publication number
|Publication type
|Application number
|Publication date
|Feb 3, 2009
|Filing date
|Dec 4, 2001
|Priority date
|Dec 4, 2000
|Fee status
|Also published as
|Publication number
|Jaroslav Gergic, Rafah A. Hosn, Jan Kleindienst, Stephane H. Maes, Thiruvilwamalai V. Raman, Jan Sedivy, Ladislav Seredi
|Original Assignee
|International Business Machines Corporation
|Export Citation
|BiBTeX, EndNote, RefMan
|Patent Citations (8), Non-Patent Citations (3), Referenced by (78), Classifications (14), Legal Events (7)
|External Links: USPTO, USPTO Assignment, Espacenet
This application is based on, and claims priority to, U.S. Provisional Application No. 60/251,085, filed on Dec. 4, 2000, which is fully incorporated herein by reference.
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by any one of the patent document or the patent disclosure, as it appears in the patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
The present invention relates generally to systems and methods for building speech-based applications and architectures and, in particular, to server-side and client-side speech frameworks employing reusable dialog components based on VoiceXML (Voice extensible Markup Language).
The computing world is evolving towards an era where billions of interconnected pervasive clients will communicate with powerful information servers. Indeed, this millennium will be characterized by the availability of multiple information devices that make ubiquitous information access an accepted fact of life. This evolution towards billions of pervasive devices being interconnected via the Internet, wireless networks or spontaneous networks (such as Bluetooth and Jini) will revolutionize the principles underlying man-machine interaction. In the near future, personal information devices will offer ubiquitous access, bringing with them the ability to create, manipulate and exchange any information anywhere and anytime using interaction modalities most suited to the an individual's current needs and abilities. Such devices will include familiar access devices such as conventional telephones, cell phones, smart phones, pocket organizers, PDAs and PCs, which vary widely in the interface peripherals they use to communicate with the user.
The increasing availability of information, along with the rise in the computational power available to each user to manipulate this information, brings with it a concomitant need to increase the bandwidth of man-machine communication. The ability to access information via a multiplicity of appliances, each designed to suit the individual's specific needs and abilities at any given time, necessarily means that these interactions should exploit all available input and output (I/O) modalities to maximize the bandwidth of man-machine communication. Indeed, users will come to demand such multi-modal interaction in order to maximize their interaction with information devices in hands-free, eyes-free environments.
VoiceXML is a markup language designed to facilitate the creation of speech applications such as IVR (Interactive Voice Response) applications. Compared to conventional IVR programming frameworks that employ proprietary scripts and programming languages over proprietary/closed platforms, the VoiceXML standard provides a declarative programming framework based on XML (eXtensible Markup Language) and ECMAScript (see, e.g., the W3C XML specifications (www.w3.org/XML) and VoiceXML forum (www.voicexml.org)). VoiceXML is designed to run on web-like infrastructures of web servers and web application servers (i.e. the Voice browser). VoiceXML is a key component for providing a voice interface to Mobile e-business. Indeed, VoiceXML allows information to be accessed by voice through a regular phone or a mobile phone whenever it is difficult or not optimal to interact through a wireless GUI micro-browser.
More importantly, VoiceXML is a key component to building multi-modal systems such as multi-modal and conversational user interfaces or mobile multi-modal browsers. Multi-modal e-business solutions exploit the fact that different interaction modes are more efficient for different user interactions. For example, depending on the interaction, talking may be easier than typing, whereas reading may be faster than listening. Multi-modal interfaces combine the use of multiple interaction modes, such as voice, keypad and display to improve the user interface to e-business. Advantageously, multi-modal browsers can rely on VoiceXML browsers and authoring to describe and render the voice interface.
There are still key inhibitors to the deployment of compelling multi-modal e-business applications. Most arise out of the current infrastructure and device platforms. Indeed, the current networking infrastructure is not configured for providing seamless, multi-modal access to information. Indeed, although a plethora of information can be accessed from servers over a communications network using an access device (e.g., personal information and corporate information available on private networks and public information accessible via a global computer network such as the Internet), the availability of such information may be limited by the modality of the client/access device or the platform-specific software applications with which the user is interacting to obtain such information. For instance, current wireless network infrastructure and handsets do not provide simultaneous voice and data access. Middleware, interfaces and protocols are needed to synchronize and manage the different channels.
Currently, application authoring methodologies are being developed to provide means to develop rich multi-modal applications. It is anticipated that most multi-modal mobile deployment will rely on wireless PDAs that can overcome the above challenges by hosting a VoiceXML browser on the client (fat client configuration) or by relying on sequential or notification-based multi-modal scenarios, where the user switches connectivity when he or she wants to interact through another modality.
Because of the inherent challenges of conversational engines (e.g., speech recognizer) that require data files (e.g., grammars), however, it is important to provide mechanisms that provide tools that hide this level of complexity. It is also important that such mechanisms and tools overcome some of the limitations imposed by VoiceXML (e.g. the VoiceXML execution model). Thus, while it is anticipated that voice (alone for multi-channel applications) and multi-modal will be key catalyst to wide adoption of mobile e-business, it is believed that such wide spread adoption of such voice and multi-modal interfaces will remain challenging until tools for building applications using voice-based reusable dialog components are available to non-speech specialists.
The VoiceXML Forum has submitted VoiceXML 1.0 to the W3C Voice Browser activity (see, e.g., W3C voice browser activity, www.w3.org/voice/). As part of its activities, the W3C Voice Browser working group has identified reusable dialog components as an item worth studying and it published a set of associated requirements (see, e.g., the W3C reusable dialog requirements for voice markup language (www.w3.org/TR/reusable-dialog-reqs)).
Accordingly, VoiceXML frameworks for reusable dialog components (server-centric and client-centric), which satisfy the published W3C reusable dialog component requirements while remaining compliant with the VoiceXML specifications, for example, would be a significant component for building voice interfaces and multi-modal applications that seamlessly operate across a plurality of channels.
The present invention relates generally to systems and methods for building speech-based applications and, in particular, to server-side and client-side speech frameworks employing reusable dialog components based on VoiceXML (Voice eXtensible Markup Language). VoiceXML reusable dialog components according to the present invention can be used for building a voice interface for use with multi-modal, multi-channel and conversational applications that offer universal access to information anytime, from any location, using any pervasive computing device regardless of its I/O modality.
In one aspect of the present invention, a method for authoring a speech application comprises the steps of:
In another aspect, a speech application server comprises:
In a preferred embodiment, a client-side reusable dialog component framework is built within the VoiceXML specifications and utilizes <subdialog> elements to call reusable VoiceXML dialog components and associated ECMAScript parameter objects to pass parameters, configuration and results. This solution is interpreted at the client side (VoiceXML browser).
In another aspect of the present invention, a server-side speech application server comprises:
Preferably, a server-side framework for reusable dialog components is based on JSP (Java Server Pages) and beans that generate VoiceXML subdialogs.
In another aspect of the present invention, the server-side and client-side frameworks for reusable VoiceXML dialog components can be combined to provide a desired solution.
In another aspect of the present invention, reusable VoiceXML dialog components are "re-entrant" to allow the reusable VoiceXML dialog components to be initiated, interrupted, inspected, and/or resumed with a partially filled result object or state object.
These and other aspects, features, and advantages of the present invention will become apparent from the following detailed description of the preferred embodiments, which is to be read in connection with the accompanying drawings.
The present invention relates generally to systems and methods for building speech-based applications and, in particular, to server-side and client-side speech frameworks employing reusable dialog components based on VoiceXML (Voice eXtensible Markup Language). VoiceXML reusable dialog components according to the present invention can be used for building speech interfaces for multi-modal, multi-channel and conversational applications that offer universal access to information anytime, from any location, using any pervasive computing device regardless of its I/O modality.
It is to be understood that the term "channel" used herein refers to a particular renderer, device, or a particular modality. Examples of different modalities/channels include speech such as VoiceXML, visual (GUI) such as HTML (hypertext markup language), restrained GUI such as WML (wireless markup language), CHTML (compact HTML), XHTML-MP (XHTML Mobile profile), and HDML (handheld device markup language) or any combination of such modalities.
The term "multi-channel application" refers to an application that provides ubiquitous access through different channels (e.g., VoiceXML, HTML), one channel at a time. Multi-channel applications do not provide synchronization or coordination across the views of the different channels.
The term "multi-modal" application refers to multi-channel applications, wherein multiple channels are simultaneously available and synchronized. Furthermore, from a multi-channel point of view, multi-modality can be considered another channel.
Furthermore, the term "conversational" or "conversational computing" as used herein refers to seamless multi-modal dialog (information exchanges) between user and machine and between devices or platforms of varying modalities (I/O capabilities), regardless of the I/O capabilities of the access device/channel, preferably, using open, interoperable communication protocols and standards, as well as a conversational (or interaction-based) programming model that separates the application data content (tier 3) and business logic (tier 2) from the user interaction and data model that the user manipulates. The term "conversational application" refers to an application that supports multi-modal, free flow interactions (e.g., mixed initiative dialogs) within the application and across independently developed applications, preferably using short term and long term context (including previous input and output) to disambiguate and understand the user's intention. Conversational application preferably utilize NLU (natural language understanding).
The following detailed description of preferred embodiments is divided into the following sections for ease of reference: Section I below provides a general description of VoiceXML reusable components according to the present invention, as well as the need, motivation and advantages of implementing frameworks based on VoiceXML reusable dialog components; Section II describes preferred embodiments of VoiceXML dialog component frameworks according to the present invention; Section III describes preferred dialog object interfaces and behaviors to support mixed initiative across subdialogs, documents and modalities using VoiceXML reusable components; and Section IV enumerates preferred specification standards for VoiceXML dialog components that fall within the W3C speech framework, as well as extensions for future platforms and standards.
I. Motivation For Employing VoiceXML Reusable Dialog Components
As noted above, specifications for reusable components are being developed within the W3C speech framework. According to the W3C reusable dialog requirements, reusable dialog components provide prepackaged functionality "out-of-the-box" that enables developers to build applications by providing standard default settings and behavior. The reusable dialog components shield developers from having to worry about many of the intricacies associated with building a robust speech dialogue, e.g., confidence score interpretation, error recovery mechanisms, prompting, etc. This behavior can be customized by a developer, if desired, to provide application-specific prompts, vocabulary, retry settings, etc.
Reusable dialog components are classified herein as "task" types and "template" types. Task components are designed to obtain some piece or pieces of information (e.g., get a date). Although task components can be configured, they will operate as-is. Template components require configuration and need to be parameterized (e.g., select a menu item, wherein the menu list must be provided.
The VoiceXML specifications identify the possibility of using declarative reusable VoiceXML pages to create a reusable library of dialogs shared among many applications. It has also been recognized that prepackaged dialogs designed to reduce the application developers effort through appropriate abstraction can occur at many levels and can be implemented in a number of different ways. For example, parameterized dialogs can be implemented as markup elements with attributes and sub-elements, as scripts built of markup elements and variables (perhaps stored in a standard library of such dialogs), or as native, pre-compiled, or otherwise non-markup language objects or modules. Therefore, from an application developer's point of view, it is advantageous to provide prepackaged reusable dialog components and sample code that can be use as libraries or as sample code/templates to build more complex applications and reusable dialog modules or customize them.
Further, it would be highly advantageous to provide dialog reusable components authored in VoiceXML. Indeed, currently, there are a variety of VoiceXML applications that are being used or developed in which VoiceXML reusable components would provide a mechanism for seamless and synchronized multi-modal interactions across a plurality of channels. Such speech applications include, for example, IVR-centric, server-centric, client-centric and embedded implementations. Further, reusable VoiceXML components (which are based on extensions of the VoiceXML standard) would enable development of mixed initiative speech applications, as well as integrated speech applications (VoiceXML) within existing web infrastructures (multi-channel, multi-modal and conversational applications).
The following sections describe frameworks according to the present invention for implementing reusable dialog components, subdialogs, and beans, which are built within VoiceXML specifications.
II. Frameworks For Reusable VoiceXML Dialog Components
In one embodiment of the present invention, reusable VoiceXML dialog components are built using VoiceXML <subdialog> elements and using associated ECMAScript parameter objects to pass parameters, configuration and results. This framework comprises a client-centric approach (e.g., VoiceXML browser) using-side reusable dialog components. The present inventions provides various frameworks for reusable dialog components built within the VoiceXML specifications.
In another embodiment, server-centric framework for implementing reusable VoiceXML components, wherein VoiceXML pages are dynamically generated to provide, e.g., dynamic manipulation or prompts, dynamic grammar compilation and dynamic access to data sources. In one preferred embodiment, a server-centric framework is based on JSP (Java Server Pages) and beans that generate VoiceXML subdialogs. In other embodiments of the present invention, a reusable VoiceXML component framework can be built using a combination of client-side and server-side reusable dialog components.
It is to be understood that the term "client-side" or "client-centric" processing as used herein refers to processing that occurs directly at the level of the presentation layer which, in the present invention, comprise a VoiceXML browser. This does not mean that the VoiceXML browser is actually physically located on the client. The VoiceXML browser may be located on the server-side, behind the telephony card, IVR or Gateway (PSTN or VoIP). Furthermore, with fat client configurations of multi-modal browsers, embedded native VoiceXML browser can appear on the client. Such configuration may introduce significant new challenges including, for example, delays, network latencies, bandwidth limitations and network overload when loading data files associated to a new VoiceXML page; and limitation of the capabilities of the local engine (500 words vocabulary), etc. These considerations are important and should not be underestimated when designing a particular solution. However, such issues are beyond the scope of this application.
It is to be further understood that the term "server-side" or "server-centric" processing refers to processing that is executed at the level of a network application server (e.g., web application server) and not at the level of an earlier piece of middleware. As a simple analogy in the web/HTML world, we would consider Javascript, Applets and static long HTML pages (or even DHTML pages) as client-side processing and CGI, JSP/Beans, ASPs and servlets as server-side processing.
Therefore, in the present case, client-side processing implies that the reusable dialog components are loaded "as is" by a Voice browser/platform to provide the voice interaction to the user. Server-side processing implies that the reusable dialog component run at the level of the web application server to contribute to the generation of the next VoiceXML page shipped and loaded by the VoiceXML browser.
Advantageously, as explained below, reusable VoiceXML component frameworks according to the present invention remain within the VoiceXML specifications and require no modification of the VoiceXML interpreter (except for features such as dynamic compilation of grammars wherein the present invention offers mechanisms for extending VoiceXML to provide such functionality).
A Reusable VoiceXML Dialog Components
A <subdialog> element invokes a "called" dialog (referred to as a subdialog) identified by its src attribute in the "calling" dialog (the src comprises the URI of the <subdialog>). A <subdialog> field item is similar to a function call: it invokes another dialog on a current page, or invokes another VoiceXML document and returns an ECMAScript Object as its result. In accordance with the present invention, subdialogs allow the reuse of a common dialog and are used to build libraries of reusable applications.
Referring now to
In the framework of
The Reusable VoiceXML dialog components 13 are invoked via the <subdialog> tag in a VoiceXML page 11. The parameter objects 16 are called from an ECMAScript <script> in the VoiceXML page 11. The VoiceXML browser 10 comprises a ECMAScript host environment. Each reusable VoiceXML dialog component 13 receives its respective ECMAScript parameter object 14 which is created as described above. Passing only one parameter in the <subdialog> simplifies the readability of the code since the amount and complexity of parameters even for a single reusable VoiceXML dialog component can be enormous. The execution follows the VoiceXML specifications.
The reusable VoiceXML dialog components 13 are implemented as standard VoiceXML documents referenced by the src attribute of the VoiceXML <subdialog>. The results of the reusable VoiceXML dialog components 13 are returned in the return variable of the associated ECMAScript parameter objects 16. Advantageously, this framework readily fits the Natural Language Semantics Markup Language for the W3C speech interface (see, e.g., www.w3.org/TR/nl-specs/).
As noted above, a library of VoiceXML reusable dialog component may reside on the server repository 15 for dynamic HTTP access at execution, or may reside locally to the VoiceXML browser 10. The local library may be downloaded and updated from the same server repository 15. The associated parameter object files are available (same name, .js extension instead of .vxml).
Task-type reusable VoiceXML dialog components can be directly used without customization/configuration. However, the components can also be downloaded in advance by the application developer for customization of the associated parameter object files and adaptation of the components. When customized, it is important to update the src arguments, especially for the parameter object files.
Template-type of reusable VoiceXML components require configuration of the associated parameter object files. Therefore, they should be downloaded, adapted and the src arguments should be updated.
It is to be appreciated that the respositories 15, 15 may further comprise emcompass default grammars and audio prompts to support the basic behavior of the reusable VoiceXML dialog components 13. Futher, the system can provide libraries or repositories of reusable ECMAScript functions that may be used in the .js files. Until supported by VoiceXML, the repository 15 or other servers can provide dynamic browser client platform.
It is to be futher appreciated that the amount of customization/adaptation that is needed can be limited by providing rich libraries expanding on the reusable components identified in the W3C reusable dialog component requirements. In addition, standardized sets of such subdialogs ("reuasable VoiceXML dialog modules") can be installed on the Voice browser side as libraries that are available to the user.
The following skeleton example illustrates the use and call of reusable VoiceXML dialog components. To begin, the following code (myComponentExample.vxml) illustrates a VoiceXML skeleton page with internal calls to a reusable VoiceXML dialog component:
It is to be appreciated that this formulation supports any type of reusable dialog components such as task or template components.
The following code (simpComponent.vxml) illustrates a corresponding VoiceXML skeleton page comprising the reusable VoiceXML dialog component:
For template as well as some task ojects, mechanisms are preferably employed to support dynamic grammar compilation. One example is illustrated below with an example reusable VoiceXMXL dialog component referred to as simpNavMenu.vxml.
The following code(simpComponent.js)illustrates a corresponding mechanism to pass parameters, configuration to and from the reusable VoiceXML dialog component:
The above myComponentExample.vxml1 illustrates the creation of an ECMAScript parameter object (mySimpComponent) for each reusable VoiceXML dialog component. The parameter objects are populated with the appropriate parameters. Again, as noted above, all parameters are optional for task reusable VoiceXML dialog components and for templaet reusable VoiceXML dialog components, additional configuration parameters are mandatory (e.g. the collection of menu items which will be rendered by a navigation menu reusable VoiceXML dialog component).
The above simpComponent.js illustrates the use of EMCAScript parameter objects as containers that provide: (i) default prompts and other object-specific resources; (ii) constructor that combines default and application-specific parameters; and (iii) common methods for manipulating parameter content.
The reusable VoiceXML dialog components are invoked via the <subdialog>tag. Each reusable VoiceXML dialog component is passed its respective ECMAScript parameter object created as described above. It is to be understood that the above example illustrates passing only one parameter in the <subdialog>for purpose of simplyfing the readability of the code but that the amount and complexity of parameters even for a reusable VoiceXML dialog component will vary. Advantageously, the execution flow follows the VoiceXML specifications.
As shown in the exemplary program code above, the reusable voiceXML dialog components are implemented as standard VoiceXML documents referenced by the src attribute of the VoiceXML <subdialog>. The simpComponent.vxml illustrates a skeleton reusable VoiceXML dialog component (simpComponent). The results of the reusable VoiceXML dialog component is returned in the return variable of the associated ECMAScript parameter object.
The following are detailed examples of the implementation of a task type (simpDate.vxml) and a template type (simpNavMenu.vxml)reusable VoiceXML dialog components.
The following VoiceXML file (MyTest.vxml) illustrates a file that calls two reusable VoiceXML dialog components:
The following code illustrates a task-type reusable Voice XML dialog component (simpDate.vxml): to collect a date input:
The following program code illustrates a template-type reusable VoiceXML dialog component (simpNavMenu.vxml) to present a select menu. As noted above, template type reusable VoiceXML dialog component require support for dynamic grammar compiliation to build on the fly grammar of the items to select in a dynamic menu.
The following code (simpDate.js) illustrates a parameter oject declaration for the simpDate reusableVoiceXML dialog component.
The following code (simpNavMenu.js) illustrates a parameter object declaration for the simpNavMenu reusable VoiceXML dialog component:
An exemplary grammar file (simpDate.gram) shown in Appendix A, specifies the associated Dategrammar file.
Preferably, to implement reusable VoiceXML dialog components within the VoiceXML specifications, a VoiceXML platform should support (i) dynamic grammar generation and (ii) passing audio as a variable. Currently, VoiceXML does not support dynamic grammar generation. In one preferred embodiment, a grammar is dynamically compiled outside of the VoiceXML interpreter. For example, <grammar src="javascript:MP.grammar"/> in simpComponent.vxml can be replaced by: <grammar src=" http://www.grammar_generator.example/dynamiccompiler?MP.grammarfURI"> or <grammar src="http://localhost/dynamiccompiler?MP.grammarfURI">, depending on whether the compiler is on a server or local.
The MP.grammarfURI is a javascript function that converts the grammar parameters in an URI compliant form. Other plug-in mechanisms may be employed.
In accordance with the present invention, it is preferred that a VoiceXML specification be added for dynamic grammar generation. Indeed, dynamic grammar generation is needed for template type reusable VoiceXML dialog components as illustrated by simpNavMenu.vxml. Moreover, there are task reusable VoiceXML dialog components that may be difficult to implement without dynamic-grammar support, e.g. spoken-and-spelled name.
VoiceXML 2.0 supports the ability to pass audio as variable. This feature is useful for implementing audio prompts. Currently, with VoiceXML 1.0, the prompts must be generated outside the VoiceXML interpreter and passed by URI similarly to the dynamic grammar case.
There are numerous advantages associated with the framework discussed above using reusable VoiceXML dialog components. One advantage is that the reusable VoiceXML dialog components are browser independent and do not require any change to existing browsers (except for the new VoiceXML requirements proposed herein). Further, the task of authoring complex reusable dialog components and modules is left to the skilled VoiceXML programmers (e.g., VoiceXML reusable dialog components, associated ECMAScript parameter object files and associated default grammars and prompts). Moreover, developers who use the reusable VoiceXML dialog components can use such components as libraries or as sample code/templates to build more complex applications and reusable VoiceXML dialog modules or customize them.
In other words, reusable VoiceXML dialog components will allow users to learn from and re-use standard components as they get deployed. This will have the same impact on easing learning and adoption as was seen with HTML where users could see how a certain page was written when authoring their sites. Similarly, the use of reusable VoiceXML dialog components will be instrumental in the growth of voice applications.
Another advantage is that the reusable VoiceXML dialog component framework described above meets the W3C VoiceXML and reusable dialog component specifications and recommendations. Further, internationalization and localization can be supported by providing different versions of the reusable VoiceXML dialog components and adapting the parameter object files accordingly. This can be provided in the above mentioned libraries or repositories or the components can easily be adapted by a developer starting from the sample reusable VoiceXML dialog components. These are examples of reusable VoiceXML dialog modules as noted above.
The simultaneous activation of the components is currently a feature beyond the scope of VoiceXML. In accordance with the present invention, however, mechanisms are provided to support this feature (e.g., mixed initiative and free flow dialog) as discussed below.
Furthermore, in a reusable VoiceXML dialog component framework as described above, return values support the NL formats, error/exception handling are explicitly addressed (as illustrated in the skeleton and examples above), prompts are set as part of the associated parameter object file and the associated error and exception handling logic is explicitly implemented in the reusable VoiceXML dialog components. The present invention supports coordination between language and component features since the reusable VoiceXML dialog components are declarative and the behavior can be immediately determined by direct examination. Further, staying within the VoiceXML framework guarantees a well predictable scoping of global commands. In addition, to provide a consistent user experience, it is preferred that a reusable dialog component framework according to the present invention follows scopes as for subdialogs. Component composition is also supported as described earlier and further because of the declarative nature of the reusable VoiceXML dialog components and modules.
Further, with respect to the W3C updated requirement proposal, reusable VoiceXML dialog components according to the present invention preferably support: (i) returning the control to the calling routine after execution of the component (by design of subdialogs); (ii) multiple invocation; (iii) multiple application can point to the same subdialog; (iv) reusable component can be configured at execution through an associated ECMAScript parameter object and no compilation is required except for grammar compilation as discussed earlier; (v) the interface is clearly standardized through the result object and the ECMAScript parameter object; and (vi) platform dependency is trivially determined by inspection of a reusable VoiceXML dialog component: if the reusable component does not contain an <object > tag, it is platform-independent but if the reusable component does contain an <object > tag, then another framework as described herein may be implemented.
The requirement for simultaneous/parallel activation of the components is more complex and is currently a feature that is currently beyond the scope of VoiceXML and currently addressed herein through shared grammars across form items. This feature is discussed in detail below (e.g., mixed initiative and free flow dialog) wherein mechanisms are added to the existing W3C requirements to support mixed initiatives: context sharing across reusable dialog components (subdialogs as well as objects).
When developing client-side VoiceXML applications, it may be difficult to build a voice-based application without access to dynamic data sources like dynamic grammar compilation and dynamic access to databases (via HTTP server or by ODBC, SQL, etc. . . . ). In accordance with the present invention, components and mechanisms are provided to access dynamic data sources.
B Reusable VoiceXML Dialog Beans
In addition to building client-side reusable VoiceXML dialog components and modules that satisfy the existing VoiceXML framework using existing VoiceXML browsers and web infrastructures, in accordance with another embodiment of the present invention, server-side solutions with dynamic generation of the VoiceXML pages are preferably employed to support dynamic manipulation of prompts, dynamic grammar compilation and dynamic access to data sources .
A common issue with server side reusable dialog components is the selection of a programming language—environment (Java, Perl, Python, PHP, C, VisualBasic, . . . ). In a preferred embodiment, the dialog component library is programming language agnostic in order to achieve wide acceptance and reusability. There are several interlingual frameworks that allow integration of software components that are implemented in different programming languages. Some of them are limited to MS Windows (COM, .NET runtime), some are limited to Linux Gnome (Bonobo), but other are platform independent (CORBA Components, UNO, XPCOM). Because inter-process communication is not necessary for the reusable dialog components, UNO and XPCOM seem to be more suitable for this task.
In general, a method for server-side generation of a VoiceXML page according to one aspect of the invention comprises the following steps. Initially, an intermediate VoiceXML code is constructed using predefined reusable dialog components 24. Then, data from the dynamic data sources 25 is inserted into the intermediate code. Finally, the resulting pure VoiceXML code (with or without <object> or <subdialog> elements) is sent to the browser 23.
It is to be appreciated that various methods may be used for implementing server-side reusable VoiceXML dialog components. For instance, server-side reusable VoiceXML dialog components may comprise VoiceXML objects that are created by reusing the client-side reusable VoiceXML dialog components and modules as described above in Section IIA. Further, in another VoiceXML framework, the VoiceXML objects may comprise beans, whereby a markup template processing engine (e.g., JSP(JavaServer Pages), ASP(Active Server Page), PHP(Personal Home Page Tools)) provides a framework to dynamically generate the VoiceXML pages using the beans. In addition, as explained below, service beans can be used to dynamically compile grammars and dynamically manipulate prompts.
In a preferred embodiment of the present invention, the server-centric framework illustrated in
The following skeleton example describes the use and call of reusable VoiceXML dialog beans. In particular, the following JSP skeleton page (component_vxml.jsp) is responsible for creating and rendering reusable VoiceXML dialog beans:
In JSP/beans framework, reusable VoiceXML dialog beans are embedded in JSP pages that are responsible for bean creation and rendering. Since a full-fledged object-oriented procedural language (Java) is used for implementing the server-side reusable VoiceXML dialog beans, the speech objects are readily reusable through inheritance or aggregation. Preferably, the rendering follows a MVC (Model-View-Controller) paradigm, where the UI renderers, acting as Views, encompass markup-specific rendering code (renderFace) and the Model maintains the modality-independent state of the object. Each bean comprises a "rendering face" for each modality supported by the server, which allows each bean to return its results (in response to a JSP request) in the appropriate modality. A Database stores the object instances and data associated with the various beans.
The "rendering face" of a given bean comprises a method that inspects (by introspection) the type of channel used for access and then, upon determining the channel type, e.g., VoiceXML access, the rendering face method returns a VoiceXML portion of the user interface to embed in the calling JSP (and in the VoiceXML pages) that the JSP dynamically generates.
More specifically, in the framework of
The exemplary CML JSP page 33 comprises a plurality of components (program code modules) such as an XFORMS portion 33 a (generally, a data model declaration), various sequences of gestures 33 b, 33 d (CML interaction pages that call gesture beans) and a sequence of service calls 33 c (which call service beans). In a hybrid case, the CML JSP page 33 may also comprise some object calls. It is to be understood that the sequence and use of such components will vary for a given CML JSP page based on the application. The XFORMS component 33 a of the CML JSP page 33 specifies one or more data models for user interaction. The XFORMS component 33 a of the CML JSP page 33 declares a data model for the fields to be populated by the user interaction that is specified by the one or more gestures 33 b, 33 d. In other words, the CML interaction page 33 b, 33 d can specify the portions of the user interaction that is binded on the data model portion 33. XForms is compatible and supports XSchema, which may be implemented herein. It is to be understood that other languages that capture data models and interaction may be implemented herein.
Those portions of the CML skeleton comprising the sequences of gestures 33 b, 33 d comprise code (scripts, tags, etc.) that access/call corresponding gesture beans in the database 32. In one embodiment, the database 32 comprises a finite set of gesture beans, wherein the set of gesture beans comprises at least one gesture bean for each of the fundamental CML conversational gestures described in the above-incorporated U.S. Ser. No. 09/544,823, and possibly a finite set of reusable dialog components that are built from the elementary finite set of gesture beans.
Furthermore, the gesture beans 32 may comprise one or more inherited gesture beans that are generated from corresponding gesture beans. An inherited gesture bean may be used to provide cosmetization or specialized rendering in a target ML associated with the corresponding gesture bean. Cosmetization or specialization is a method for optimizing an application for a given channel (device, modality or browser) or a class of channel (e.g., Nokia cell phones, etc.). For example, specialization may includes providing a background for a page, changing the layering of a page into frames, fragmenting a WML document across multiple deck of cards, specifying the voice characteristics for a TTS prompt or an audio prompt to play back, changing the message to present to the user when spoken versus the displayed message, skipping a gesture not needed in a given modality, etc. This concept is analogous to cosmetized XSL rules for the conversational gestures as described in the above-incorporated U.S. Ser. No. 09/544,823.
Next, those portions of the CML JSP page 33 that correspond to sequences of service calls 33 c comprise program code that are specified as service bean calls. Preferably, the programming model for the service beans is imperative (i.e., the conventional model), although any suitable constrained model may be implemented. In addition, unlike the set of gesture beans which is preferably limited, the number of service beans that may be employed is unlimited. The service beans are employed to, e.g., provide access to dynamic data, provide access to the backend legacy content, provide a mechanism to maintain dialog states, etc. Server beans comprise a rendering component which, as explained below, allows the service call to be rendered in CML (and possibly a XFORMS) format. In other words, the service beans preferably produce CML and/or XFORMS page portions (including possible calls to gesture beans) that are inserted in the resulting CML-JSP page prior to rendering of such page.
There are various advantages associated with a framework based on reusable VoiceXML dialog beans as shown in
Further, the use of the MVC principle enables extension of the application authoring to multi-channel and multi-modal applications. In addition, a beans framework and server side application can maintain context and enable context sharing and dialog flow control. Therefore, this framework is ready to support mixed initiative applications when available.
Another advantage is that reusable VoiceXML dialog beans will allow users to learn from and re-use standard components as they get deployed, which will lead to the growth of voice applications.
Further, no extensions are required for voice browser or VoiceXML with the server-side VoiceXML generation approach and the reusable VoiceXML dialog beans discussed above. A reusable VoiceXML dialog bean framework satisfies the requirements and recommendations of the W3C speech framework.
III. Dialog Object Interfaces and Behaviors
This section addresses the use of the <object> tag within Voice XML for encapsulating dialog elements and other service calls whose behavior is implemented via platform-specific objects, and enumerates some requirements to ensure that the component framework used for dialog objects enables rich context sharing amongst various dialog components comprising a conversational application. It is to be understood that this discussion goes beyond the current VoiceXML 2.0 specifications and execution model.
In accordance with an embodiment of the present invention, a framework is provided for context sharing objects and subdialogs to support their parallel activation and mixed initiative across objects, subdialogs and documents. This functionality is preferably achieved by using "re-entrant" objects and or subdialog, meaning that objects and subdialogs can be initiated, interrupted, inspected and resumed with a partially filled result/state object. These frameworks require extensions of the VoiceXML specification, the form interpretation algorithm and the VoiceXML execution model. These frameworks provide support for advanced mixed initiative beyond current VoiceXML specification that allow mixed initiative for fields of a same form or within a same document.
In the VoiceXML framework, the <object> element is used to expose platform-specific functionality for use by a VoiceXML application. A<param> element can be used to pass parameters to the <object> when it is invoked. When an <object > is executed, it returns an ECMAScript result object as the value of its form item variable.
The use of reusable VoiceXML dialog components as described in the previous section support standard VoiceXML implementations. In view of the reusable VoiceXML dialog component frameworks presented above, it is preferred that the use of binary objects (i.e., non-declarative VoiceXML) be limited as much as possible to the implementation of functions or behaviors not supported by VoiceXML so as to maintain browser interoperability. For example, <object > can be used to implement speaker recognition functions as enrollment, verification and identifications, and other functions not currently supported by VoiceXML.
Similarly, it is preferred that dialog modules which only use functions supported by VoiceXML rely on the reusable VoiceXML dialog component and module framework described above or the server-side beans framework to remain within the VoiceXML specifications.
A. Object Types
There are various type of objects that are classified as "service objects" and "interaction objects". Service objects comprise objects that do not process the dialog with the user to affect the dialog, but can rather affect the state of the dialog, the state of the application or the state of the platform (client or server) or they can process the dialog with no direct impact on this dialog other than confirmation of success, failure or completion. Examples of service objects include, e.g., objects that check the caller ID, check the time of the call, pre-populate a dialog form based on information known about the user or record and log a whole conversation (as compared to the <record>) function.
Service objects also provide dynamic access to data sources (via HTTP server or by ODBC, SQL, etc . . . ). The required functionality of dynamic data source access objects makes it virtually impossible to develop such access objects using VoiceXML. Therefore, consideration should be given to using some native programming language—preferably the same as the language that the VoiceXML browser is written in. It is also possible to imagine these objects as similar to browser plug-ins.
The current VoiceXML <object> specifications seem adequate to support such services objects. Service objects are similar to the service beans as described above with reference to
Further, interaction objects comprise objects that directly process dialogs with the user. Interaction objects can be further classified into "dialog module" objects and "I/O processing" objects. Dialog module objects comprise objects that provide partial dialog and dialog logic to populate a set of fields. They can provide functions analogous to the reusable VoiceXML dialog components or modules as defined above, except that their internal execution model does not have to be limited to the VoiceXML execution model and capabilities. For example, a speech biometric verification dialog module can provide verification of a user simultaneously based on acoustic verification (speaker verification) and content verification (acceptance of the answers to questions).
It is to be noted that the reusable VoiceXML dialog components (as well as the beans) discussed above is analogous to the dialog module object type, using the <subdialog> tag instead of <object>. Therefore, in accordance with one embodiment of the present invention, the expected behavior of both elements is closely linked.
I/O processing objects comprise objects that process user inputs and output to the user and populate a set of fields without carrying any dialog or partial dialog with the user or process <param> to produce and output to the user. Examples of I/O processing objects include, e.g., a speaker identification I/O processing object that performs text-dependent or text-independent speaker identification or an NLG (Natural Language Generation) processing object that generates the prompt to feed to a TTS engine on the basis of a set of attribute value pairs through <param>.
Interaction objects bring more demanding requirements. Because of the similarity between interaction objects and the subdialogs as described above in the reusable VoiceXML dialog component framework, we will discuss issues with execution flow, the form interpretation algorithm, etc. for both <subdialog> and <object> tags.
Preferably, to support mixed initiative according to the present invention, interaction objects and subdialogs preferably support parallel activation (e.g., objects that can process the next I/O event). Consequently, in accordance with the present invention, mechanisms are preferably implemented to (i) decide what object or objects process a given event (ii) share events and state (context) between objects and to (iii) switch between running objects and pass from one dialog to another (e.g., Stop or interrupt an object and save its state, access to state, launch an object in a prescribed state, etc.). This goes beyond the level of parallel activation described in the updated W3C reusable dialog component requirements, where parallel activation would not play beyond the first prompt or entry point in the object.
Further, to support mixed initiative, interaction objects and subdialogs preferably support the use of expert systems to improve processing of particular dialog situations and therefore base a portion of an interaction object on the result of another object that was running in parallel.
In addition, some level of support is preferred for multi-channel and multi-modal rendering. For example, with the JSP/bean approach, the use of the MVC principle and a renderFace function enables to reuse the same beans across multiple channel (multi-channel applications) with possible synchronization of these channels (multi-modal applications): renderFace (out, getModality ( ), cp); where getModality ( ) returns the access channel(s) and cp describes the client profile (this can be based on the W3C CC/PP working group deliverables).
B. Execution and Navigation Flow Issues
To illustrate the impact of the above requirements, consider the VoiceXML mixed initiative specifications—to make a form mixed initiative, where both the computer and the human direct the conversation, the form must have one or more <initial> form items (which controls the initial interaction in a mixed initiative form) and one or more form-level grammar (which allows the fields of a form to be filled in any order and allows more than one field to be filled as a result of a single utterance).
This works well for <field> as form items. It does not work for binary <object> (e.g., interaction objects) or <subdialog> as form items, when mixed initiative is expected across objects (instead of intra-objects). Indeed, as specified in the VoiceXML forum, once entered, an object must complete before returning the hand to the VoiceXML interpreter.
Indeed, the existing VoiceXML specifications only indicate that if a form has form-level grammars, its fields can be filled in any order and that more than one field can be filled as a result of a single user utterance. Further, the form's grammars can be active when the user is in other dialogs. If a document has two forms on it and both forms have grammars that are active for that document, a user could respond to a request from one form with information about the other, and thus direct the computer to talk about the second form instead. The user can speak to any active grammar, and have fields set and actions taken in response.
This mechanism enables mixed initiative only within a document and relies on the "scope" of active grammars to decide the form item activated by a given user input. However, this mechanism does not provide for mixed initiative across documents, nor does it provide a mechanism for exiting form items that have their own execution flow before completion of the dialog flow.
We have identified various issues with the current VoiceXML specifications:
Further issues of the current VoiceXML specifications are that the <record> and <transfer> behavior is uncertain when mixed initiative is to be supported such functions can not be escaped by the user (for example during bridging, before returning to the interpreter), or combined with another query. Further, with respect to <block> behavior, it is unclear whether or not there can be mixed initiative between block form items and the other form items of the parent form.
The execution model for objects is not specified in the current version of VoiceXML 2.0 as they are essentially platform specific. Because objects are a priory binaries, not available to inspection by the VoiceXML interpreter, it is not possible to simply extend the grammar scope and execution flows as we could do with subdialogs.
C. Re-Entrant Objects
Therefore, in accordance with the present invention, to support mixed initiative, objects and subdialogs (e.g., the next generation of reusable VoiceXML dialog components or modules) are preferably "reentrant" or support "re-entrant" behavior and interfaces.
In accordance with one aspect of the present invention, a re-entrant object or subdialog is one that supports launch with a partially filled state. More specifically, an ECMAScript result object can be manipulated prior to launching the object and partially filled. Further, the object initiated in its partially filled state (i.e. result object), has an execution flow that allows it to continue to collect the missing members of its result object. A result object according to the present invention differs from the current specification since, e.g., the result object may comprise a description of the full relevant internal state of the object (rather than just the final return values).
Further, a re-entrant object or subdialog is one that supports an execution_flag that prescribes if the object should:
Further, when the event approach is followed, the re entrant object or subdialog preferably can launch blocking events, be blocked, support set and get on the members of its result object and allow its execution to be resumed with a modified partially filled result object and continue to collect the missing members of its result object.
In addition a re-entrant object or re-entrant subdialog is one that allows possible I/O sharing through an additional object, e.g., allowing an utterance be passed to other engines associated to other objects for separate processing when needed. This is important if parallel objects use different engines or different data files and properties. This can be done as member of the result object.
Re-entrant objects should now throw not only error.unsupported.object if the particular platform-specific object is not supported, but throw an analogous error.unsupported.object.execution_flag, if the selected execution flag is not supported. Vendors may decide to revert to a default behavior (probably the default execution_flag settings) or wait for the interpreter to handle.
It is to be appreciated that the <subdialog> specifications do not have to be fundamentally modified. To support re-entrance, the present invention adds an execution_flag argument and modifies the execution flow when the execution_flag is not set at its default value. Because of the declarative nature of subdialogs, the manipulation of the result object and capability to handle modified or partially filled result objects can be readily supported with an upgraded VoiceXML interpreter.
There are various advantages to using re-entrant objects and subdialogs. For example, mixed initiative across and between subdialogs, objects and documents becomes possible when supported by the VoiceXML interpreter. Another advantage is that parallel and simultaneously objects and subdialogs can be appropriately supported.
A further advantage is that a user input can be processed through parallel engines. This enables, for example, a speech biometric implementation as described earlier where acoustic and content based authentication are simultaneously performed. Re-entrance also enables to decide on the fly which of a set of parallel engines should process a given input.
It is further possible to involve other expert resources to handle dialog errors in an object or subdialog or condition internal processing of an input on the result of other parallel objects or subdialogs. In addition, context-sharing between modalities, including explicit time-sharing of input events in different modalities becomes possible for multi-modal applications.
Experimental Results
To validate the proposed framework, we have developed a set of re-entrant reusable dialog components using the IBM form-based dialog manager (FDM) as described in the references by Papineni, et al. "Free-flow dialog management using forms", Proc. Eurospeech, Budapest, 1999, and Davies, et al., "The IBM conversational telephony system for financial applications", Proc. Eurospeech, Budapest, 1999. The set of components essentially covers the set of reusable dialog requirements for Voice Markup language (www.w3.org/TR/reusable-dialog-reqs) and a series of transaction-level reusable dialog components. In this setting, the VoiceXML browser simultaneously loads components (<object>) that are loaded with a same activation scope as argument. This goes beyond the current VoiceXML specifications. As VoiceXML does not really specify context sharing, this implementation limits context sharing and mixed initiative between simultaneously activated objects; not between objects and the VoiceXML application. This is a simplification of the application, clearly not a limitation of the framework. Actually, we separately extended support of context sharing between the VoiceXML page and active objects. However, this has architectural implications that go beyond the scope of this application. Components were activated sequentially or in parallel. We did not develop a framework that enables object -oriented composition of the components. This is just a limitation of the FDM as currently implemented; again not a limitation of the framework.
Our implementation demonstrated the sufficiency of the framework to support parallel activation of dialog components. On the other hand, we did not really confirm the usefulness for application developers of a limited set of re-entrant reusable component as in the W3C reusable dialog requirement. Indeed, the use of transaction level re-entrant reusable components quickly appeared more efficient. This may just be because we did not allow building the transaction-level components from the foundation set in the W3C requirements. Because of the commonalties between objects and subdialogs, we consider that this also validated our requirements on VoiceXML subdialogs.
IV. Implementation Summary
With respect to the VoiceXML specifications and dialog reusable components as described herein, it is preferred that support for client-side dynamic grammar compilation be added to the VoiceXML specification. Further, it is preferred that any reusable dialog component framework follows scopes as for subdialogs. In addition, it is preferable to limit the use of binary objects, i.e. non VoiceXML declarative, to the implementation of functions or behaviors not supported by VoiceXML. In addition, dialog modules that only use functions supported by VoiceXML should implement the reusable VoiceXML dialog component and module framework, or its server side bean version, as described herein. Moreover, a mechanism is preferably provided for sharing and aggregating the parent form grammar with its subdialogs. Block behavior under mixed initiative should also be specified.
With respect to future extensions of VoiceXML, service objects are preferably implemented in addition to interaction objects. Such service object should enable dynamic data source access mechanisms (via HTTP (Hypertext Transfer Protocol) servers or by ODBC (Open Database Connectivity), SQL (Structured Query Language), etc). Further, objects and subdialogs are preferably re-entrant or support a re-entrant behavior and interfaces. Moreover, extensions of the VoiceXML interpreter and Form Interpretation Algorithm (FIA) are provided to handle modified or partially filled result subdialogs and objects and support mixed initiative across subdialogs, objects, documents and modalities.
Although illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the present system and method is not limited to those precise embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the scope or spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as defined by the appended claims.
Appendix A: simpDate.gram
grammar com.ibm.speech.vxml.grammars_en.date;
// Copyright (c) 2000 IBM Corp. All Rights Reserved.
// The return value is a string, in the format yyyymmdd, yyyymm??, yyyy????,
????mmdd, or ????mm??.
// The grammar will return a "?" in the appropriate positions if the user omits
either the year or day.
public <date> = <dayofmonth> <of_yr> <yr>
this.$value = $yr + $dayofmonth;
this.$tts = $dayofmonth.$tts + ", " + $yr.$tts;
this.$value = "????" + $dayofmonth;
this.$tts = $dayofmonth.$tts;
<mo> <of_yr> <yr>
this.$value = $yr + $mo + "??";
this.$tts = $mo.$tts + " " + $yr.$tts;
<yr_of> <yr>
this.$value = $yr + "????";
this.$tts = $yr.$tts;
<mo_of> <mo>
this.$value = "????" + $mo + "??";
this.$tts = $mo.$tts;
this.$tts = $other.$tts;
<dayofmonth> = ( <month29> [the] <ordinal01to29> | [the] <ordinal01to29> [day]
of <month29> )
this.$value=$month29 + $ordinal01to29;
this.$tts=$month29.$tts + " " + $ordinal01to29.$tts;
( <month30> [the] <ordinal01to30> | [the] <ordinal01to30> [day]
of <month30>)
this.$value=$month30 + $ordinal01to30;
this.$tts=$month30.$tts + " " + $ordinal01to30.$tts;
( <month31> [the] <ordinal01to31> | [the] <ordinal01to31> [day]
of <month31>)
this.$value=$month31 + $ordinal01to31;
this.$tts=$month31.$tts + " " + $ordinal01to31.$tts;
<month29> = february
{ this.$value="02"; this.$tts="February" };
<month30> = april
{ this.$value="04"; this.$tts="April"; }
| june
{ this.$value="06"; this.$tts="June"; }
| september
{ this.$value="09"; this $tts="September"; }
| november
{ this.$value="11"; this.$tts="November"; };
<month31>= january
{ this.$value="01"; this.$tts="January"; }
| march
{ this.$value="03"; this.$tts="March"; }
| may
{ this.$value="05"; this.$tts="May"; }
| july
{ this.$value="07"; this $tts="July"; }
| august
{ this.$value="08"; this.$tts="August"; }
| october
{ this.$value="10"; this.$tts="October"; }
| december
{ this.$value="12"; this.$tts="December"; }
<yr> = <teens> hundred
this.$value = $teens.$value + "00" ;
this.$tts = $teens.$value + "00";
| <teens> ( <NULL> | hundred [and] ) <2dNum1to99>
this.$value = $teens.$value + $2dNum1to99;
this.$tts = $teens.$value + $2dNum1to99.$value;
| ninety <numbers1to9>
this.$value = "199" + $numbers1to9;
this.$tts = "199" + $numbers1to9.$value;
| two thousand [and] <2dNum1to99>
this.$value = "20" + $2dNum1to99;
this.$tts = "20" + $2dNum1to99.$value;
| two thousand (zero | oh | nil | null | ought) <numbers1to9>
this.$value = "20" + "0" + $numbers1to9;
this.$tts = "20" + "0" + $numbers1to9.$value;
| two thousand
this.$value = "2000";
this.$tts = "2000";
| <2dNum1to99> <numbers01to99>
this.$value = $2dNum1to99 + $numbers01to99;
this.$tts = $2dNum1to99.$value + $numbers01to99.$value;
| twenty <numbers01to99>
this.$value = "20" + $numbers01to99;
this.$tts = "20" + $numbers01to99.$value;
<mo> = <month29>
{ this.$tts = $month29.$tts }
| <month30>
{ this.$tts = $month30.$tts }
| <month31>
{ this.$tts = $month31.$tts } ;
<other> = <NULL>
{ var months = new Array("January", "February", "March",
"April", "May", "June",
"July", "August", "September", "October",
"November", "December");
var days = new Array("First", "Second", "Third", "Fourth",
"Fifth", "Sixth", "Seventh", "Eighth", "Ninth",
"Tenth", "Eleventh", "Twelfth",
"Thirteenth", "Fourteenth", "Fifteenth", "Sixteenth", "Seventeenth",
"Eighteenth", "Nineteenth",
"Twentieth", "Twenty First", "Twenty
Second", "Twenty Third", "Twenty Fourth", "Twenty Fifth", "Twenty Sixth",
"Twenty Seventh", "Twenty Eighth", "Twenty Ninth",
"Thirtieth", "Thirty First");
var daysInMonth = new
var currtime = new Date();
var yr = currtime.getFullYear();
var mo = currtime.getMonth() +1;
var da = currtime.getDate();
if ((yr % 4) == 0)
daysInMonth[1] = 29;
( today
var da_str = new String((da < 10) ? ("0" + da.toString()) :
var mo_str = new String((mo < 10) ? ("0" + mo.toString()) :
this.$value = yr + mo_str + da_str;
this.$tts = months[mo−1] + " " + days[da−1] + ", " + yr;
| yesterday
yr = ((mo == 1) && (da == 1)) ? yr−1 : yr;
mo = (da == 1) ? (mo−1) : mo;
if (mo == 0) mo = 12;
var dam1 = (da == 1) ? (daysInMonth[mo−1]) : (da−1);
var dam1_str = new String((dam1 < 10) ? ("0" +
dam1.toString()) : dam1.toString());
var mo_str = new String((mo < 10) ? ("0" + mo.toString()) :
this.$value = yr + mo_str + dam1_str;
this.$tts = months[mo−1] + " " + days[dam1−1] + ", " + yr;
| tomorrow
yr = ((mo ==12) && (da ==31)) ? yr+1 : yr;
var mo0 = mo;
mo = (da == daysInMonth[mo−1]) ? (mo+1) : mo;
if (mo==13) mo = 1;
var dap1 = (da == daysInMonth[mo0−1]) ? 1 : (da+1);
var dap1_str = new String((dap1 < 10) ? ("0" +
dap1.toString()) : dap1.toString());
var mo_str = new String((mo < 10) ? ("0" + mo.toString()) :
this.$value = yr + mo_str + dap1_str;
this.$tts = months[mo−1] + " " + days[dap1−1] + ", " + yr;
// [in] [the year] [of] -all combinations except: "in of"
<of_yr> = ( in | in the year | in the year of | the year of | the year | of |
<mo_of> = (in | in the month | in the month of | the month of | the month |
<yr_of> = (in | in the year | in the year of | the year of | the year | <NULL> );
<ordinal2to9> = second
{ this.$value="2"; this.$tts="Second"; }
| third
{ this.$value="3"; this.$tts="Third"; }
| fourth
{ this.$value="4"; this.$tts="Fourth"; }
| fifth
{ this.$value="5"; this.$tts="Fifth"; }
| sixth
{ this.$value="6"; this.$tts="Sixth"; }
| seventh
{ this.$value="7"; this.$tts="Seventh"; }
| eighth
{ this.$value="8"; this.$tts="Eighth"; }
| ninth
{ this.$value="9"; this.$tts="Ninth"; };
<ordinal1to9> = first
{ this.$value="1"; this.$tts="First"; }
| <ordinal2to9>
{ this.$value=$ordinal2to9;
this.$tts=$ordinal2to9 $tts; };
<ordinal10to19> = tenth
{ this.$value="10"; this.$tts="Tenth"; }
| eleventh
{ this.$value="11"; this $tts="Eleventh"; }
| twelfth
{ this.$value="12"; this $tts="Twelfth"; }
| thirteenth
{ this.$value="13"; this.$tts="Thirteenth"; }
| fourteenth
{ this.$value="14"; this $tts="Fourteenth"; }
| fifteenth
{ this.$value="15"; this.$tts="Fifteenth"; }
| sixteenth
{ this.$value="16"; this.$tts="Sixteenth"; }
| seventeenth
{ this.$value="17"; this $tts="Seventeenth"; }
| eighteenth
{ this.$value="18"; this.$tts="Eighteenth"; }
| nineteenth
{ this.$value="19"; this.$tts="Nineteenth"; };
<ordinal2to19> = <ordinal2to9>
{ this.$value=$ordinal2to9;
this.$tts=$ordinal2to9.$tts; }
| <ordinal10to19>
{ this.$value=$ordinal10to19;
this.$tts=$ordinal10to19.$tts; };
<ordinal02to19> = <ordinal2to9>
{ this.$value="0" + $ordinal2to9;
this.$tts=$ordinal2to9.$tts; }
| <ordinal10to19>
{ this.$value=$ordinal10to19;
this.$tts=$ordinal10to19.$tts; };
<ordinal01to19> = first
{ this.$value="01"; this.$tts="First"; }
| <ordinal02to19>
{ this.$value=$ordinal02to19;
this.$tts=$ordinal02to19.$tts, };
<ordinaltens20to90> = twentieth
{ this.$value="20"; this.$tts="Twentieth"; }
| thirtieth
{ this.$value="30"; this.$tts&"Thirtieth"; }
| fortieth
{ this.$value="40"; this.$tts="Fortieth", }
| fiftieth
{ this.$value="50"; this.$tts="Fiftieth"; }
| sixtieth
{ this.$value="60"; this.$tts="Sixtieth"; }
| seventieth
{ this.$value="70"; this.$tts="Seventieth"; }
| eightieth
{ this.$value="80"; this.$tts="Eightieth"; }
| ninetieth
{ this.$value="90"; this.$tts="Ninetieth"; };
<ordinal2to100> = <ordinal2to19>
| <ordinaltens20to90>
{ this.$value=$ordinaltens20to90; }
| <tens> <ordinal1to9>
{ this.$value=$tens + $ordinal1to9; }
| [one] hundredth
{ this.$value="100"; };
<ordinal1to100> = first
{ this.$value="01"; this.$tts="First"; }
| <ordinal2to100>
{this.$value=$ordinal2to100; };
<ordinal01to29> = <ordinal01to19>
{ this.$value=$ordinal01to19;
this.$tts=$ordinal01to19.$tts; }
| twentieth
{ this.$value="20";
this.$tts="Twentieth"; }
| twenty <ordinal1to9>
{this.$value="2" + $ordinal1to9;
this.$tts="Twenty " + $ordinal1to9.$tts; };
<ordinal01to30> = <ordinal01to29>
this.$tts=$ordinal01to29.$tts; }
| thirtieth
{ this.$value="30"; this.$tts="Thirtieth";
<ordinal01to31> = <ordinal01to30 >
{ this.$value=$ordinal01to30,
this.$tts=$ordinal01to30.$tts; }
| thirty first
{ this.$value="31". this.$tts="Thirty
First"; };
// The following were moved from the number grammar
// (some of these are duplicated in the time grammar - if you change one here,
change the other as well) :
= <tens20to90> <numbers1to9>
{ this.$value$tens20to90 +
$numbers1to9; }
| <tens>
{ this.$value=$tens + "0"; }
| <teens>
| <numbers1to9>
{ this.$value="0" + $numbers1to9;
<numbers01to99> = <numbers02to99>
| (zero | oh | nil | null | ought) one { this.$value="01"; };
<numbers02to99> = (zero | oh | nil | null | ought) <numbers2to9>
{this.$value="0" +
$numbers2to9; }
| <teens>
| <tens>
{ this.$value=$tens + "0"; }
| <tens> <numbers1to9>
{ this.$value=$tens + $numbers1to9;
= one
{ this.$value="1"; }
| <numbers2to9>
<numbers2to9> = two
{ this.$value="2"; }
| three
{ this.$value="3"; }
| four
{ this.$value="4"; }
| five
{ this.$value="5"; }
| six
{ this.$value="6"; }
| seven
{ this.$value="7"; }
| eight
{ this.$value="8"; }
| nine
{ this.$value="9"; };
= eleven
{ this.$value="11"; }
| twelve
{ this.Svalue="12"; }
| thirteen
{ this.$value="13"; }
| fourteen
{ this.$value="14"; }
| fifteen
{ this.$value="15"; }
| sixteen
{ this.$va1ue="16"; }
| seventeen
{ this.$value="17"; }
| eighteen
{ this.$value="18"; }
| nineteen
{ this.$value="19"; };
= ten
{ this.$value="1"; }
| twenty
{ this.Svalue="2"; }
| thirty
{ this.$value="3"; }
| forty
{ this.$value="4"; }
| fifty
{ this.$value="5"; }
| sixty
{ this.$value="6"; }
| seventy
{ this.$value="7"; }
| eighty
{ this.$value="8"; }
| ninety
{ this.$value="9"; };
= twenty
{ this.$value="2"; }
| thirty
{ this.$value="3"; }
| forty
{ this.$value="4"; }
| fifty
{ this.$value="5"; }
| sixty
{ this.$value="6"; }
| seventy
{ this.$value="7"; }
| eighty
{ this.$value="8"; }
| ninety
{ this.$value="9"; };
// --- end of rules that were moved from number.gram and possibly duplicated in
time.gram ---------------------
|Cited Patent
|Filing date
|Publication date
|May 25, 1999
|May 29, 2001
|Lucent Technologies Inc.
|Method and apparatus for assembling and presenting structured voicemail messages
|Oct 2, 1998
|Jul 31, 2001
|Motorola, Inc.
|Voice browser for interactive services and methods thereof
|Mar 31, 2000
|Jan 1, 2002
|Siebel Systems, Inc.
|Web client-server system and method for incompatible page markup and presentation languages
|Dec 15, 1999
|Jul 23, 2002
|Nokia Corporation
|Voice packet data network browsing for mobile terminals system and method using a dual-mode wireless connection
|Feb 9, 2000
|Dec 3, 2002
|Cisco Technology, Inc.
|Arrangement for defining and processing voice enabled web applications using extensible markup language documents
|Apr 28, 2000
|Jun 10, 2003
|Cisco Technology, Inc.
|Browser-based arrangement for developing voice enabled web applications using extensible markup language documents
|Mar 14, 2000
|Aug 5, 2003
|Lucent Technologies Inc.
|Web-based voice dialog interface
|Apr 9, 1999
|Oct 21, 2003
|Inroad, Inc.
|System and process for voice-controlled information retrieval
|U.S. Appl. No. 09/544,823, filed Apr. 6, 2000, entitled Methods and Systems for Multi-Modal Browsing and Implementation of a Conversational Markup Language.
|U.S. Appl. No. 09/837,024, filed Apr. 18, 2001, entitled Systems and Methods for Providing Conversational Computing Via Javaserver Pages and Javabeans.
|Voice XML Forum, "Voice eXtensible Markup Languag (VoiceXML) versin 1.0", May 5, 2000. available at <"http://www.w3.org/TR/2000/Note-voicexml-20000505">.
|Citing Patent
|Filing date
|Publication date
|Dec 19, 2003
|Feb 9, 2010
|At&T Intellectual Property Ii, L.P.
|Method and apparatus for automatically building conversational systems
|Aug 10, 2005
|Dec 7, 2010
|Nuance Communications, Inc.
|Overriding default speech processing behavior using a default focus receiver
|Mar 28, 2007
|Jan 11, 2011
|Nuance Communications, Inc.
|Partially filling mixed-initiative forms from utterances having sub-threshold confidence scores based upon word-level confidence data
|Jul 13, 2006
|May 10, 2011
|Elias Assad
|System and method for communications over a computer network
|Dec 20, 2005
|Jul 26, 2011
|International Business Machines Corporation
|System and method for emulating the processing of java server pages
|Nov 5, 2004
|Mar 6, 2012
|Vocollect, Inc.
|Voice-directed portable terminals for wireless communication systems
|Dec 22, 2009
|May 8, 2012
|At&T Intellectual Property Ii, L.P.
|Method and apparatus for automatically building conversational systems
|Oct 21, 2005
|Jul 24, 2012
|Nuance Communications, Inc.
|Creating a mixed-initiative grammar from directed dialog grammars
|Nov 12, 2009
|Feb 26, 2013
|Vocollect Healthcare Systems, Inc.
|Training/coaching system for a voice-enabled work environment
|May 7, 2012
|Jun 11, 2013
|At&T Intellectual Property Ii, L.P.
|Method and apparatus for automatically building conversational systems
|Aug 29, 2006
|Jul 23, 2013
|Intellectual Ventures I Llc
|System for embedding programming language content in XML
|Nov 9, 2006
|Jul 23, 2013
|Siemens Enterprise Communications Gmbh & Co. Kg
|Method for triggering at least one first and second background application via a universal language dialog system
|Oct 12, 2007
|Oct 29, 2013
|International Business Machines Corporation
|Methods and systems for authoring of mixed-initiative multi-modal interactions and related browsing mechanisms
|Aug 27, 2010
|Jan 21, 2014
|Research In Motion Limited
|Sending a communications header with voice recording to send metadata for use in speech recognition, formatting, and search mobile search application
|Jul 22, 2010
|Feb 25, 2014
|Vocollect, Inc.
|Method and system for correctly identifying specific RFID tags
|Jun 11, 2013
|May 6, 2014
|At&T Intellectual Property Ii, L.P.
|Method and apparatus for automatically building conversational systems
|Aug 1, 2008
|Sep 16, 2014
|Vlingo Corporation
|Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility
|Oct 1, 2007
|Nov 4, 2014
|Vlingo Corporation
|Application text entry in a mobile environment using a speech processing facility
|Aug 1, 2008
|Nov 11, 2014
|Vlingo Corporation
|Using speech recognition results based on an unstructured language model in a mobile communication facility application
|Jan 21, 2010
|Nov 11, 2014
|Vlingo Corporation
|Dealing with switch latency in speech recognition
|Aug 25, 2010
|Dec 16, 2014
|Unwired Planet, Llc
|Web browser implementation of interactive voice response instructions
|Feb 24, 2014
|Jan 13, 2015
|Vocollect, Inc.
|Method and system for correctly identifying specific RFID tags
|Oct 21, 2009
|Feb 3, 2015
|Vlingo Corporation
|Internal and external speech recognition use with a mobile communication facility
|Aug 27, 2010
|Feb 3, 2015
|Vlingo Corporation
|Multiple web-based content category searching in mobile search application
|Oct 1, 2007
|Mar 31, 2015
|Vlingo Corporation
|Speech recognition text entry for software applications
|Dec 22, 2004
|Jul 14, 2015
|Nuance Communications, Inc.
|Enabling voice selection of user preferences
|Dec 19, 2014
|Sep 20, 2016
|Vocollect, Inc.
|Method and system for correctly identifying specific RFID tags
|Nov 10, 2014
|Oct 4, 2016
|Nuance Communications, Inc.
|Dealing with switch latency in speech recognition
|Nov 10, 2014
|Nov 15, 2016
|Nuance Communications, Inc.
|Dealing with switch latency in speech recognition
|Sep 10, 2010
|Mar 21, 2017
|Vocollect, Inc.
|Multimodal user notification system to assist in data capture
|Dec 15, 2014
|Apr 11, 2017
|Nuance Communications, Inc.
|Multiple web-based content category searching in mobile search application
|Dec 19, 2003
|Jun 23, 2005
|At&T Corp.
|Method and apparatus for automatically building conversational systems
|Dec 22, 2004
|Jun 22, 2006
|New Orchard Road
|Enabling voice selection of user preferences
|Jul 13, 2006
|Jan 18, 2007
|Elias Assad
|A system and method for formatted inter-node communications over a computer network.
|Aug 10, 2005
|Feb 15, 2007
|International Business Machines Corporation
|Overriding default speech processing behavior using a default focus receiver
|Oct 21, 2005
|Apr 26, 2007
|International Business Machines Corporation
|Creating a Mixed-Initiative Grammar from Directed Dialog Grammars
|Aug 29, 2006
|May 24, 2007
|Ben Franklin Patent Holding Llc
|System for embedding programming language content in XML
|Dec 20, 2005
|Jun 21, 2007
|International Business Machines Corporation
|System and method for emulating the processing of Java Server Pages
|Mar 2, 2006
|Oct 4, 2007
|Wally Brill
|System and method for creating designs for over the phone voice enabled services
|May 8, 2006
|Nov 8, 2007
|International Business Machines Corporation
|Method and system for automatically discovering and populating a palette of reusable dialog components
|Oct 12, 2007
|Feb 7, 2008
|Healey Jennifer A
|Methods and Systems for Authoring of Mixed-Initiative Multi-Modal Interactions and Related Browsing Mechanisms
|Oct 1, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile environment speech processing facility
|Oct 1, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile environment speech processing facility
|Oct 3, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile content search environment speech processing facility
|Oct 1, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile environment speech processing facility
|Oct 3, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile navigation environment speech processing facility
|Oct 3, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile messaging environment speech processing facility
|Oct 3, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile local search environment speech processing facility
|Oct 3, 2007
|Sep 11, 2008
|Cerra Joseph P
|Mobile browser environment speech processing facility
|Mar 28, 2007
|Oct 2, 2008
|International Business Machines Corporation
|Partially filling mixed-initiative forms from utterances having sub-threshold confidence scores based upon word-level confidence data
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using speech recognition results based on an unstructured language model in a mobile communication facility application
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using speech recognition results based on an unstructured language model with a navigation system
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Adapting an unstructured language model speech recognition system based on usage
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Tagging speech recognition results based on an unstructured language model for use in a mobile communication facility application
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using an unstructured language model associated with an application of a mobile communication facility
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using contextual information for delivering results generated from a speech recognition facility using an unstructured language model
|Aug 1, 2008
|Jan 29, 2009
|Cerra Joseph P
|Using speech recognition results based on an unstructured language model with a music system
|Feb 20, 2009
|Aug 27, 2009
|Prakash Somasundaram
|Voice-activated emergency medical services communication and documentation system
|Mar 11, 2005
|Oct 8, 2009
|Glaxo Group Limited
|Novel m3 muscarinic acetylchoine receptor antagonists
|Jul 15, 2009
|Feb 11, 2010
|Nu Echo Inc.
|Methods and Systems for Providing Grammar Services
|Nov 9, 2006
|Feb 18, 2010
|Hans-Ulrich Block
|Method for Triggering at Least One First and Second Background Application via a Universal Language Dialog System
|Dec 22, 2009
|Apr 22, 2010
|At&T Corp.
|Method and Apparatus for Automatically Building Conversational Systems
|Oct 21, 2009
|Apr 29, 2010
|Phillips Michael S
|Internal and external speech recognition use with a mobile communication facility
|Jan 21, 2010
|Jul 22, 2010
|Meisel William S
|Dealing with switch latency in speech recognition
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Utilizing user transmitted text to improve language model in mobile dictation application
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Sending a communications header with voice recording to send metadata for use in speech recognition and formatting in mobile dictation application
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Transmitting signal quality information in mobile dictation application
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Multiple web-based content search user interface in mobile search application
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Command and control utilizing content information in a mobile voice-to-speech application
|Aug 27, 2010
|Mar 3, 2011
|Phillips Michael S
|Multiple web-based content category searching in mobile search application
|Aug 27, 2010
|Mar 17, 2011
|Phillips Michael S
|Sending a communications header with voice recording to send metadata for use in speech recognition, formatting, and search in mobile search application
|Mar 7, 2011
|Jun 30, 2011
|Verizon Patent And Licensing Inc.
|Generating voice extensible markup language (vxml) documents
|Aug 25, 2010
|Mar 1, 2012
|Openwave Systems Inc.
|Web browser implementation of interactive voice response instructions
|Nov 4, 2014
|Feb 26, 2015
|Unwired Planet, Llc
|Generation of natively implementable instructions based on interactive voice response instructions
|Feb 20, 2008
|Nov 9, 2010
|Vocollect Healthcare Systems, Inc.
|Body-worn mobile device
|Aug 20, 2010
|Aug 9, 2011
|Vocollect Healthcare Systems, Inc.
|Body-worn mobile device
|Aug 19, 2010
|Aug 16, 2011
|Vocollect Healthcare Systems, Inc.
|Body-worn mobile device
|U.S. Classification
|715/234, 704/10, 704/246, 704/235, 704/260
|International Classification
|Cooperative Classification
|European Classification
|Jul 15, 2002
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GERGIC, JAROSLAV;HOSN, RAFAH A.;KLEINDIENST, JAN;AND OTHERS;REEL/FRAME:013097/0441;SIGNING DATES FROM 20020430 TO 20020529
|Sep 17, 2012
|Maintenance fee reminder mailed
|Jan 28, 2013
|Surcharge for late payment
|Jan 28, 2013
|Fee payment
Year of fee payment: 4
|Sep 16, 2016
|Maintenance fee reminder mailed
|Feb 3, 2017
|Lapse for failure to pay maintenance fees
|Mar 28, 2017
|Expired due to failure to pay maintenance fee
Effective date: 20170203


text:
Extrusion of Whey Proteins
Extrusion technology is utilized in many areas of the food industry, including the production of snack foods, breakfast cereals, animal feed, and other diverse food products. The Utah State University Dairy Products Lab is home to a bench-scale APV Baker twin screw cooker- extruder, which is available for small-scale production and research. The extruder is configured to produce both puffed and texturized products.
In 2003, researches at Utah State University patented both the process, and products of extruded whey, including a high protein puffed product, and a texturized meat analog, now licensed to Grande Cheese of Lomira Wisconsin (US patent 6,607,77). Optimizing and characterizing these texturized whey products has been a very fruitful area of research, and many applications have yet to be explored. Possible areas for further research include:

subset: commoncrawl


text:
Dialectical Behavioral Therapy
Archived Responses:
May 2007
Can anyone recommend or know about a Dialectical Behavioral Therapy (DBT)-trained therapy practice in the East Bay, preferably Berkeley/Albany area? Searching
I know a wonderful therapist who practices in Albany on Solano Ave. and specializes in DBT. She is warm, sensitive and intelligent and has many years of experience. She works with individuals using DBT. Her name is Marianne Davis, MFT and her phone number is 510-928-2455. Barbara
DBT Sorry I don't know anyone in the East Bay, however, UCSF has A DBT clinic for adolescents. There may be referrals if you call there as they train therapists in addition to the clinic practice (i understand the clinic is always waitlisted, and the therapists rotate) pp


text:
APG-L ArchivesArchiver > APG > 2009-07 > 1248897319
From: "Bonnie Kohler" <>
Subject: Re: [APG] APG PMC (Little Rock) Syllabus?
Date: Wed, 29 Jul 2009 15:55:19 -0400
References: <firstname.lastname@example.com><66968A228F154C07B671FFF6BA4009FF@acer511eba12df><firstname.lastname@example.com><77755EEAA5B54666B6D08F0B15B5EA88@acer511eba12df><email@example.com><email@example.com><email@example.com><email@example.com><FA39CA2015F3458190A0B1497A17D139@acer511eba12df><firstname.lastname@example.com><firstname.lastname@example.com>
Craig wrote:
> Hmmm, so they want virtual conferences do they?
I was really taken with the Library of Congress Webinar that gave a tour of
the LOC's online content. I liked the interactivity, the white board, the
e-mails that participants sent in. The presenter had an assistant who posted
the URLs to Websites that were mentioned. I don't recall seeing the
presenter; I only remember hearing her voice.
I didn't save the transcript of the event, feeling that it would be too
text-intense to be searchable. I didn't retain much of what I heard. I have
a vague notion of how to search more effectively at the site. I think I
needed a take-away outline as a reminder. I couldn't take notes because the
pace was quick, and I was following the directions of navigating to
different portions of the Website.
For some reason, I think the lecture would have had a more lasting impact if
I had seen the presenter. Perhaps it's the visage that stimulates my memory.
In regard to Second Life, I have a vague notion of what an avatar is. I
suspect an avatar is not as visually complex as a human.
My $.02.
Bonnie Dunphy Kohler
APG Florida Chapter member
This thread:
|Re: [APG] APG PMC (Little Rock) Syllabus? by "Bonnie Kohler" <>


text:
I had rhino/septo-plasty over 1.5 years ago, and am unhappy with the results: (1) Pointy/Pinched tip of the nose that is droopy, especially when smiling--near hook appearance. The pic shown is right profile WITHOUT smiling. The tip droops much further down and looks awful when I smile. (2) Hanging and scarred columella. Prominent scar line. Skin is bulgy and hangs lower after scar line. I can email more pictures directly. Any opinions on corrective measures very much appreciated.
Am I a Good Candidate for Revision Rhinoplasty?
Doctor Answers 4
Candidate for Revision Rhinoplasty
You are a candidate for rhinoplasty revision. It is difficult to make specific recommendations because animation makes your nose look worse. Ideally, you need to be examined by an experienced surgeon - another alternative is to submit additional pictures.
Revision for droopy tip
There are options, through cartilage grafting and establishing tip support, to keep your nose from drooping as much and to correct a pointy and pinched tip
Learn more: http://www.seattlerhinoplasty.com/html/index.php
Revision rhinoplasty should be carefully considered
Revision rhinoplasty is not something to undergo lightly. The revision rhinoplasty procedure involves removal of any residual dorsal hump that is present which can be composed of both bone and cartilage. The reason the tip droops upon animation is due to a ligament which connects the tip of the nose to the orbicularis oris muscle. The smiling muscle when contracted will pull the tip down. To prevent this from occurring the ligament is divided at the basal spine of the nose and this stops the depressor septi ligament action. This surgery can all be done through a closed rhinoplasty technique without an open rhinoplasty, thus avoiding a scar to the columella. Columellar show is reduced by trimming both the membranous and cartilaginous septum.
Learn more: http://www.seattlefacial.com
You might also like...
Revision rhinoplasty can help downturned noses
As the sub-specialty of revision Rhinoplasty matures, along with the specialists in revision surgery, we are getting better at recognizing the reasons why people need revision surgery. When the tip of the nose still points down after rhinoplasty and even worse with smiling it is often due to the tip cartilages having been at a higher angle than average before the first surgery. This is called cephalic oriented lateral crura. What that means is that if the upper part of the tip cartilages are angled too high before surgery, unless compensated for the tip of the nose will continue to appear to plunge after.
Treatment for tips that still plunge is by either supporting the nostrils with grafts, relocating the tip cartilages lower down so it helps thrust the tip upward or both. I suggest you seek a specialist in revision rhinoplasty to get a personal evaluation.
Learn more: http://revisionrhinoplastyny.com
These answers are for educational purposes and should not be relied upon as a substitute for medical advice you may receive from your physician. If you have a medical emergency, please call 911. These answers do not constitute or initiate a patient/doctor relationship.


text:
The Weinstein Co's "My Week with Marilyn" had its premiere at Film Society of Lincoln Center in New York the other night. Starring the superb (and likely Oscar-nomination-worthy) Michelle Williams as the icon of blonde, the film also features Kenneth Branagh as Sir Laurence Olivier, Dougray Scott as Arthur Miller, Judi Dench, Dominic Cooper, Julia Ormond, and actor and Burberry model Eddie Redmayne as Colin Clark.
The story (slated for release on November 4, 2011) has it that Colin Clark escorted Marilyn Monroe through the English countryside for a week, away from the prying eyes of the world's media attention, and in the absence of her then husband, playwright Arthur Miller.
When Celebs.com's Editor caught up with Michelle Williams back in April, she admitted to recent experience checking out "Twilight" with her friends. Check out the interview above, and the crazy flashbulbs below, I've kept the footage of the photographers to less than a minute, but Michelle probably stood there for more than 2 minutes – would you be able to handle it? To say that Michelle was being very accommodating seems an understatement, and we can't wait to see the film.
Footage courtesy of The Weinstein Co.


text:
Zuo Modern Mars Chair For Sale
Modern leather chair from Zuo.... and it Swivels!!!
Petal Lounge Chair by Zuo Modern:The Petal Lounge Chair is a reproduction of the Corona Chair. Very cool conversational chair for a home with a modern style.Tags: ottomans, chairs, lounge chairs, modern furniture, zuoAdded 7 Years Ago from Cartprice
Bent plywood seat covered in Jazz Stripe stained wood veneer seat. Very modern ... very cool!
Very cool modern chair. The seat is plastic and the base is chrome. It swivels.Tags: furniture, modern, chairs, modern chairs, zuo, neptune chairAdded 7 Years Ago from Cartprice


text:
Colorado water history to get its closeup in The Great Divide
In the devastating aftermath of our recent floods, it's easy to forget that water has typically been a scarce resource in Colorado, battled over by competing interests, even as its rivers feed into eighteen states, from the Pacific Ocean to the Mississippi. But The Great Divide, a recently announced documentary film project, promises to be a revealing journey through the state's water heritage and its role in supplying the headwaters for much of the West.
Expect some spectacular aerial footage, lively storytelling -- and water, water everywhere.
"We try to find an interesting historical thread," says Jim Havey, the project's director, "and there's nothing that tells the story of Colorado better than water."
Havey started Havey Productions, a Denver-based documentary house, more than thirty years ago. The company has produced a string of striking, informative films about Colfax Avenue, Union Station, and other historical subjects that have aired on local public television or in schools and libraries. But with sponsorship by Colorado Humanities (and other fundraising still underway), Havey is shooting to draw a national audience for The Great Divide, figuring that the history of Colorado's waterways -- and sharp discussion of different approaches to conserving and exploiting the resource, from the ancient pueblos to the acequias of the San Luis Valley to the big dam battles of the twentieth century -- comprises an important chapter in the development of the West.
Havey's films aren't politically charged works of activism, so don't expect a lot of tirades about the politics and injustices of water law. But do anticipate a well-informed group of talking heads, such as the University of Colorado's Patricia Limerick and Colorado Supreme Court justice Gregg Hobbs, discussing the turning points in the state's water development and their impact. "We've got a really stellar lineup of experts," Havey says.
Echo Park.
Agriculture, recreation, development and conservation interests will all get their say, with particular attention to some of the critical moments, such as the nixing of the Two Forks project and the decision to forego the proposed water diversion in northwest Colorado's Echo Park in favor of Glen Canyon Dam, that have shaped the state's much-engineered waters. Havey is shooting for a release date in the spring of 2015 -- which should give everyone a chance to dry out and get thirsty.
More from our Environment archive: "Photos: The 100 dirtiest power plants in America -- including two in Colorado."
Get the ICYMI: Today's Top Stories Newsletter Our daily newsletter delivers quick clicks to keep you in the know
Catch up on the day's news and stay informed with our daily digest of the most popular news, music, food and arts stories in Denver, delivered to your inbox Monday through Friday.
Recommended for You
Powered by SailThru
Upcoming Events
Denver Broncos vs. Green Bay Packers
TicketsSat., Aug. 26, 7:00pm
Denver Broncos v. Green Bay Packers - HALF PRICE GAME
TicketsSat., Aug. 26, 7:00pm
Colorado Rockies vs. Detroit Tigers
TicketsMon., Aug. 28, 6:40pm
Comedian Michael Coylar
TicketsFri., Aug. 18, 7:30pmpowered by goldstar
Sponsor Content


text:
RPR Soundsystem, Pearson Sound & Morphosis (Live) photo gallery
Nick Ensing
Ticket sales have ended
RPR Soundsystem
Petre Inspirescu
Craig Richards
Pearson Sound
Radioactive Man (live)
Terry Francis
Doc Martin
Distinguished Romanian trio Rhadoo, Raresh and Petre Inspirescu return to Farringdon as the RPR Soundsystem this Saturday. It's fast becoming one of our favourite long-haul occasions when we get to witnesses the trio engage the Room One floor all night in an journey that revels in the spectacular sound all of their own.
The ever-creative personality of Craig Richards returns to Room Two working at doing what he loves most: performing and playing alongside the artists that truly inspire him. The Nothing Special is the perfect vehicle for that and this time he's invited adventurous UK producer Pearson Sound of Hessle Audio to make his second appearance to date on a Saturday night and Lebanese live act Morphosis, who produced one of his more prominent albums last year with What Have We Learned on Delsin Records.
Making the trip from his native LA shores, Doc Martin lands in Room Three to show you why his name is synonymous with new, fresh and exciting underground sounds. Immersing himself in the world of music he loves most he'll be playing next to our resident Terry Francis in his continuing exploration of a wide collection of records.
Sat, 29 Jul 2017
1221 attending
Sat, 29 Jul 2017
539 attending
TBA - East London
Sat, 5 Aug 2017
5329 attending
Sat, 5 Aug 2017
1057 attending
Sun, 6 Aug 2017
1197 attending
Sat, 12 Aug 2017
11366 attending
Sat, 19 Aug 2017
3810 attending
Sun, 20 Aug 2017
2340 attending


text:
- Packages ▼
- Bookings ▼
Explore ▼
- Ideas
- Themes
- Events
- More
- Login
- Plan A Trip
- 08061122222
- Post FREE Package
Getting drunk and out is not always abominable unless you are not bumping into others or driving back home! Well, critics might draw the daggers to chop this observation into pieces; but that is not at all meant for them. A contrasting side is a natural phenomena in every object in the world we live in. This is summer time in many parts of the world and despite the occasional rain, the mercury is soaring high all the time. And what all we crave for in these times is a cool refreshing mug of any beverage- be it alcoholic or a mocktail. Another thing you can do is to chalk out one holiday destination that is relatively colder than the rest of the world. Of all the countries you browse through make sure you emphasize on Sweden. Being a Nordic country, Sweden is all what you want. Now let's just combine the beverage mug and Sweden. The remains or the outcome will be some of the most fabulous bars you have ever visited. The capital Stockholm is the hotspot for these urber cool bars.
Out and out high end, this is a wine bar that has attained an iconic status over the time for its league of high profile guests. The sheer architecture of the bar is spell binding with an optimum blend of ancient decor and modern aspects. Advance booking of seats are suggested as this one is much busier place, despite the vaulting price.
This amazing bar is a part of the S dra Teatern theatre that is synonymous with S dermalm (a district in central Stockholm). But even if you are not attending any dramatic performances, you can step in here. The drinks you will get here are prepared with utmost care and is like a piece of art that comes out of the theatre and the snacks are totally ethnic. You can let your head do a slight swing to the music played in the background.
This is a Bar just opposite as to its name but Hell's Kitchen is meant for those bar hoppers who want the party bel going on and on. You have all the peppy numbers being played and the thrill of dance goes on till dawn.
The bar under title seems to be a scene drawn out from a medieval book with the aura of the time filling up the air. The Carmen raises from the grave and celebrates the time gone past here with only candle light arrangement and soothing Medieval music. The bar attendants are also clad in the garments reminiscent of that era. Not only the drinks, don't miss out on the lip smacking dishes.