text:
|Publication number
|Publication type
|Application number
|Publication date
|4 Sep 2001
|Filing date
|7 Jun 1993
|Priority date
|28 Mar 1991
|Fee status
|Also published as
|Publication number
|Jerome Rene Bellegarda, Dimitri Kanevsky
|Original Assignee
|International Business Machines Corporation
|Export Citation
|BiBTeX, EndNote, RefMan
|Patent Citations (20), Non-Patent Citations (23), Referenced by (107), Classifications (13), Legal Events (4)
|External Links: USPTO, USPTO Assignment, Espacenet
This application is a continuation of copending application Ser. No. 07/856,803 filed on Mar. 24, 1992, which is a continuation of copending application Ser. No. 07/676,601.
This invention relates generally to message recognition method and apparatus and, in particular, to improved message recognition method and apparatus that integrates handwriting recognition and speech recognition.
Due to the difficulties inherent in the analysis of complex processes such as speech and handwriting, the machine recognition of these two methods of communication has thus far met with limited success.
The separate development of speech and handwriting recognition methodology has resulted in two alternative means to improve the effectiveness of a man-machine interface, particularly in the context of workstations and personal computers. These separately developed approaches both attempt to utilize natural ways of communicating, specifically voice and handwriting, in order to establish an unconstrained dialogue with a computer.
One rational for developing Automatic Speech Recognition (ASR) is to provide a dictation machine capable of reducing or eliminating the need for a keyboard. More generally, ASR can be defined as any system that uses as input a set of patterns produced by a speaker. ASR may also be extended to include systems that recognize speech based on the measurement of acoustic vibrations in the throat and chest, as well as systems which recognize the movements of lips, jaws, etc. In this regard reference is made to an article entitled "Automatic Lipreading to Enhance Speech Recognition", CH2145-1, pps. 40-47, IEEE (1985), by E. D. Petajan.
Similarly, one rational for developing Automatic Handwriting Recognition (AHR) has also been directed towards reducing or eliminating the requirement for the conventional computer keyboard. Traditionally, AHR recognizes handwritten characters captured in real time on an electronic tablet.
By example, in human factor studies there are often conducted large numbers of psychological experiments to measure a user's reaction to some parameters of interest. The majority of such experiments involve many participants, each of whom must register their responses for later analysis. This analysis is usually accomplished automatically through special-purpose software. It is of paramount importance that the subject's reaction to the measured parameters be isolated as completely as possible from their reaction to the logistics of the experiments. In particular, experiments involving moderately unnatural interfaces, such as a keyboard, may conceivably distract the attention of the subject away from the essence of the experiments. Also, the modus operandi for some experiments may not leave the subject sufficient time to type a response.
In this situation, the only reasonable alternatives are to dictate the responses into a tape recorder or to write them by hand on an answer sheet. Unfortunately, the use of such natural interfaces tends to complicate the interpretation of the experiments. Converting the subject's responses into a form suitable for automatic analysis requires subsequent effort by clerical personnel or by the participants themselves. All dictated/handwritten material must be carefully typed and then thoroughly reviewed for typing errors, lest the results be biased in some unpredictable way. The required strenuous proof-reading requires a substantial amount of time and causes large delays in interpreting experiments.
One method to avoid these delays without sacrificing the naturalness of the interface is to have the participants directly produce soft copies of their responses, either by dictating into an ASR or by providing handwriting to an AHR. Both of these approaches, however, suffer from the deficiencies enumerated below.
One disadvantage inherent in conventional ASR and AHR techniques is that an acceptable recognition rate is often achieved only in a user-dependent mode with isolated (word or character) inputs. Operating in this way results in several undesirable constraints, including: (a) a user must train the system before being able to use it; and (b) a user must interact with the system in a slightly unnatural way, either by speaking with pauses between words or by writing with spaces between letters.
Another disadvantage inherent in conventional ASR and AHR techniques is that 100% recognition accuracy is extremely difficult to achieve, due to an inability of a human to be perfectly consistent over time in the manner in which he or she speaks or writes.
This disadvantage also imposes several constraints including: (a) a user must identify any errors that may be introduced during decoding, which tends to distract the user's attention away from the subject of the message; and (b) a user must correct wrongly decoded words or characters which, as a matter of practicality, requires the use of an alternative technology, such as a keyboard.
The following U.S. Patents are cited as being of interest to the teaching of the invention.
Commonly assigned U.S. Pat. No. 3,969,700, issued Jul. 13, 1976, entitled "Regional Context Maximum Likelihood Error Correction for OCR, Keyboard, and the Like", to Bollinger et al., describes a general error correction scheme, applicable to pattern recognition problems, that use segmentation prior to recognition, including optical character recognition (OCR) and ASR. No specific mention is made of on-line AHR and there is no disclosure of employing both ASR and AHR in a message recognition system.
U.S. Pat. No. 4,651,289, issued Mar. 17, 1987, entitled "Pattern Recognition Apparatus and Method for Making Same", to Maeda et al., describes an algorithm whose primary goal is to introduce robustness in a template matching procedure, such as is used in ASR and AHR. To this end there are employed multiple templates per pattern to be recognized, rather than a single template. Scores from multiple matches are appropriately combined to arrive at a final decision. There is no disclosure of applying this method simultaneously to speech and handwriting.
U.S. Pat. No. 4,736,447, issued Apr. 5, 1988, entitled "Video Computer", to Korsinsky describes a multi-facet interface which is said to acquire a variety of data, including speech and handwriting. The basic recognition algorithm utilized is template matching. No attempt is made to integrate the diverse sources of information, either for recognition purposes or otherwise. Also, there is no mention of error-correction.
U.S. Pat. No. 4,774,677, issued Sep. 27, 1988, entitled "Self-Organizing Circuits", to Buckley, describes a class of algorithms and architectures pertinent to the general field of pattern recognition. However, as in the previous references, the issue of the integration of ASR and AHR is not addressed.
U.S. Pat. No. 4,907,274, issued Mar. 6, 1990, entitled "Intelligent Work Station", to Nomura et al., describes a telephone interface incorporating speech recognition and synthesis. The application involves image compression as well, but there is no discussion of AHR or OCR.
Other art of interest includes the following.
European Patent Application 0 355 748 describes a fast classification algorithm which is intended to be used in conjunction with a hierarchical approach to pattern recognition. The domain of applications appears to be OCR. There is no explicit mention of speech or of an integration of ASR and AHR.
IBM Technical Disclosure Bulletin Vol. 22, No. 5, October 1979, entitled "Handwriting/Speech Signal Analyzer" by J. Gould et al. describes an analyzer to discriminate between pause times and production times in on-line AHR or ASR.
An article entitled "Experiments on Neural Net Recognition of Spoken and Written Text" IEEE Transaction on Acoustics, Speech and Signal Processing, Vol. 36, No. 7, July 1988 by D. Burr compares two different classes of algorithms for use in pattern recognition problems such as AHR and ASR. The combined use of speech and handwriting is not considered, and error correction is not addressed.
It is thus one object of the invention to provide a message recognition system that employs a functional complementarity of speech and handwriting inputs.
It is another object of the invention to provide method and apparatus for sequentially using a handwriting or a speech input in an error correcting technique for the other.
It is another object of the invention to provide method and apparatus that integrates AHR and ASR in a message recognition system to exploit the complementarity of speech and handwriting inputs as simultaneous sources of information.
It is another object of the invention to provide method and apparatus that integrates AHR and ASR in a message recognition system to achieve a combined use of speech and handwriting recognition that significantly improves the overall accuracy of an automatic message recognizer.
It is one further object of the invention to provide method and apparatus that integrates AHR and ASR in a message recognition system in (a) a sequential manner to achieve error correction, (b) in a simultaneous manner using separate ASR and AHR components, and (c) in an simultaneous manner using a single ASR/AHR component.
The foregoing and other problems are overcome and the objects of the invention are realized by method and apparatus that employs an integrated use of speech and handwriting recognition to improve an overall accuracy, in terms of throughput, of an automatic recognizer. The invention exploits a complementarity of acoustic and stroke information to provide method and apparatus having improved message recognition accuracy relative to either speech recognition or handwriting recognition alone.
In accordance with a method of the invention, and apparatus for carrying out the method, of operating an automatic message recognition system, the following steps are executed: a user's speech is converted to a first signal; a user's handwriting is converted to a second signal; and the first signal and the second signal are processed to decode a consistent message conveyed separately by the first signal and by the second signal or conveyed jointly by the first signal and the second signal.
The step of processing includes the steps of converting the first signal into a plurality of first multi-dimensional vectors and converting the second signal into a plurality of second multi-dimensional vectors. For a system employing a single (combined) speech and handwriting component the step of processing includes a further step of combining individual ones of the plurality of first multi-dimensional vectors and individual ones of the plurality of second multi-dimensional vectors to form a plurality of third multi-dimensional vectors.
The multi-dimensional vectors are employed to train the parameters of a set of combined word models, for a combined use of speech and handwriting, or to train two sets of word models, for separate use during speech and handwriting recognition. During use, the probabilities, or likelihoods, of each input feature vector are computed using the appropriate set(s) of word models to generate a list or lists of candidate words. A candidate word is selected as representing a message based upon criteria including the probability score and, for the separate-use message recognition system, two weighting factors derived during training of the models.
The integrated use of ASR and AHR has been found to be advantageous for several reasons. Firstly, there exists complementarity in information. That is, information gathered through speaking and information gathered through handwriting complement one another. Secondly, there is provided complementarity in performance. That is, the manner in which speech and handwriting recognition algorithms operate complement one other. Thirdly, there is provided complementarity in functionality. That is, those functions most readily implemented using ASR and AHR complement each other. For example, AHR exhibits editing capabilities which are more difficult to implement using ASR while some functions, such as generating a large body of text, can be implemented more efficiently through a speech input.
The above set forth and other features of the invention are made more apparent in the ensuing Detailed Description of the Invention when read in conjunction with the attached Drawing, wherein:
FIG. 1 is a block diagram of a message recognition system that is constructed and operated in accordance with a first embodiment of the invention to include a separate use of speech and handwriting information;
FIG. 2 is a block diagram of a message recognition system that is constructed and operated in accordance with a second embodiment of the invention to include an integrated use of speech and handwriting information;
FIG. 3 provides examples of four different types of handwriting that may be input to the system;
FIG. 4a illustrates a handwritten character being processed to generate a feature vector for a point (P);
FIG. 4b illustrates a four-dimensional handwriting feature vector generated for the point (P) by collecting a plurality of spatial attributes;
FIG. 4c illustrates, in the frequency domain, a speech sample being processed to obtain a feature vector therefrom, specifically, the determination of amplitudes associated with a plurality of frequency bands, not necessarily all of equal bandwidths;
FIG. 4d illustrates a twenty-dimensional speech feature vector generated from the speech sample;
FIG. 4e illustrates one embodiment of a combined speech and handwriting feature vector, specifically, a concatenation of the handwriting feature vector of FIG. 4b and the speech feature vector of FIG. 4d;
FIG. 4f illustrates another embodiment of a combined feature vector, specifically, a concatenation of the speech feature vector of FIG. 4d with another form of handwriting feature vector that is generated from FIG. 4a by performing a frequency analysis similar to FIG. 4c; and
FIG. 5 is a flowchart illustrating the operation of the message recognition system of the invention.
Referring to FIG. 1 there is illustrated in block diagram form a message recognition system 10 constructed and operated in accordance with a first embodiment of the invention. As employed herein a message is considered to be information generated by a human subject and entered into the system 10, the information being entered through an acoustic signal, a handwritten stroke signal, or through a combination of acoustic and stroke signals. The information may represent individual characters, including numbers, letters of the alphabet, punctuation and the like. The information may also represent words, phrases, and paragraphs.
System 10 includes a user interface 12 coupled to a display terminal 14 for displaying information to a user of the system 10. A pointing device 16, such as a mouse and/or a keyboard, may be provided for inputting additional information to the processor 12. In this regard it is noted that for an application employing the method and apparatus of the invention the primary in-flow of information is preferably via a combination of speech and handwriting inputs, and not through the pointing device if one is provided.
In accordance with the invention the system 10 includes an Integrated Speech and Writing Interface (ISWI) 18. This dual input user interface is comprised of a handwriting transducer 20, typically an electronic tablet and stylus, and a speech transducer 22, typically embodied as a microphone coupled to an analog to digital converter.
It is noted that the handwriting transducer may be embodied in any means suitable for detecting a motion of a writing implement during use by a human subject. Also, the speech transducer 22 may include any means suitable for generating a set of patterns expressive of speech. By example, the above mentioned article entitled "Automatic Lipreading to Enhance Speech Recognition", CH2145-1, pps. 40-47, IEEE (1985), by E. D. Petajan, discusses a solid state camera that images a subject's lips and nostrils during speech. The author states that a combination of acoustic and visual recognition candidates yields a final speech recognition accuracy which greatly exceeds acoustic recognition accuracy alone.
Handwritten characters generally fall into the four groups depicted in FIG. 3, the groups being depicted in increasing order of recognition complexity. Specifically, these groups include a first type of writing (W1) known as box discrete wherein individual characters are formed within predefined areas, or boxes, thereby simplifying the task of character segmentation. A second type of writing (W2) is known as spaced discrete wherein the user intentionally forms each character such that no character touches another. A third type of writing (W3) is known as run-on discrete wherein the user is not constrained and may form characters that touch, or "run-on" to, one another. A fourth type of writing (W4) is cursive writing which presents the most complex segmentation task of the four styles depicted in FIG. 3.
In regard to stroke segmentation and identification, reference is made to the following commonly assigned U.S. Patents as providing examples of various methods of handwritten character recognition: U.S. Pat. No. 4,024,500, issued May 17, 1977, entitled "Segmentation Mechanism for Cursive Script Character Recognition Systems" (N. M. Herbst and J. H. Morrissey); U.S. Pat. No. 4,727,588, issued Feb. 23, 1988, entitled "System for Automatic Adjustment and Editing of Handwritten Text Image" (A. S. Fox, E. C. Greanias, J. Kim, and C. C. Tappert); and U.S. Pat. No. 4,731,857, issued Mar. 15, 1988, entitled "Recognition System for Run-on Handwritten Characters" (C. C. Tappert).
Reference is also made, by example, to the teaching of IBM Technical Disclosure Bulletin, Vol. 27, No. 9, 1985, "Handwriting Recognition by Substroke Segmentation and Composition" (J. M. Kurtzberg and C. C. Tappert) for showing a technique for recognizing cursive script that is handwritten upon an electronic tablet. Reference is also made to an article entitled "Segmentation Procedure for Handwritten Symbols and Words" IBM Technical Disclosure Bulletin, Vol. 25, No. 7B, December 1982 by J. M. Kurtzberg and C. C. Tappert, and to an article entitled "Flexible, Space-Time Segmenter for Handwritten Words" IBM Technical Disclosure Bulletin, Vol. 29, No. 11, April 1987 by A. S. Fox, J. Kim and C. C. Tappert.
With the use of a conventional electronic tablet for the handwriting transducer 20 data is typically acquired at 70 points per second. The sampled handwriting is applied to an associated handwriting feature vector (FVH) processor 24. As employed herein a feature vector is a set of characteristics of the input accumulated during a specific time interval. Processor 24 generates one or more handwriting feature vectors for each input stroke. This is preferably accomplished, as indicated in FIGS. 4a and 4 b, in reference to a center of gravity (CG) of a stroke. The input stroke, by example a lower-case "a", is captured by the electronic tablet. Each feature vector includes elements expressive of an x and a y displacement of a sample point (P) on the stroke, relative to the center of gravity, a displacement (b) of P relative to a baseline (BL), and angular information (theta) relating to a line (L) tangent to P. The resulting feature vector, for this presently preferred embodiment, has four elements and thus represents a four dimensional (4-d) vector.
With the use of a microphone and A/D converter as the speech transducer 22 the speech is typically sampled at a 20 kHz rate and is Fast Fourier transformed. The amplitude (a) within 20 frequency bands is determined as (a1, . . . a20) for each speech sample.
The following articles are cited as teaching aspects of speech sampling and transformation that are suitable for use by the system 10: J. R. Cohen, "Application of an Auditory Model to Speech Recognition", J. Acoustic. Soc. of America, Vol. 85, No. 6, pp. 2623-2629, June 1989; A. Nadas, D. Nahamoo, and M. A. Picheny, "Speech Recognition Using Noise-Adaptive Prototypes", IEEE Trans. Acoustics, Speech, Signal Processing, Vol. 37 No. 10, pp. 1495-1503, October 1989; and "Adaptive Labelling: Normalization of Speech by Adaptive Transformations Based on Vector Quantization", in Proc. 1988 Int. Conf. on Acoustics, Speech, Signal Processing, New York, N.Y. pp. 40-43, April 1988.
FIG. 4c illustrates a speech sample being processed to obtain a speech feature vector (FVS) therefrom. The determination of amplitudes associated with the plurality of frequence bands yields a twenty dimensional (20-d) frequency domain feature vector shown in FIG. 4d. The output of the speech transducer 22 is applied to an associated FVS processor 26 that generates the series of 20-d speech feature vectors, wherein each element of the vector is expressive of an amplitude within one of the 20 sampling bands.
During training of the system 10 the feature vectors are each applied to an associated word model training processor. Specifically, the handwriting feature vectors are applied to a handwriting model trainer (MTH) processor 28 and the speech vectors are applied to a speech model trainer (MTS) processor 30. Each of the model trainer processors is coupled to the user interface 12 for interacting with a user during training. This interaction may involve the entry of training sets of sentences by the user and the display of training results. Each model trainer processor functions to determine a set of optimal parameters for an associated model, specifically a handwriting word model (WMH) 32 and a speech word model (WMS) 34. In addition, a function of each of the training processors 28 and 30 is to determine optimal weights (W) 37, represented as α and β, that govern the relative contribution of speech and handwriting to the final message recognition. For example, if during training on a specific set of training data speech is found to provide a higher likelihood, or score, than handwriting, the weight associated with the speech likelihood is increased and the weight associated with the handwriting likelihood is decreased. If the two likelihood scores are found to be comparable the weights are set to be the same, i.e. 0.5. The use of the weights is described in greater detail below.
In a presently preferred embodiment of the invention the speech trainer 30 operates in accordance with a hidden Markov model technique to identify phonemes and individual words. By example, for English speech there are four general classes of phonemes: nasals, stops, vowels and fricatives. The training and use of Markov models for speech recognition is disclosed in a journal article entitled "A Maximum Likelihood Approach to Continuous Speech Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-5, No. 2, March 1983 pp. 179-190 by L. Bahl et al. The technique described in this article is one suitable technique for implementing the speech language model and model trainer. In general, a hidden Markov model or source is comprised of a collection of states connected to one another by transitions which produce symbols from a finite alphabet. For training, one state is chosen as an initial state and another state is chosen as a final state. A training algorithm assigns probabilities to all strings of transitions from the initial state to the final state, as well as all outputs generated in the process.
The role of the Markov word model for speech is to represent the pronunciation of a word. The parameters of the Markov model are trained by relating the acoustic signal produced by the speaker to the word uttered. It should be noted that whereas there is one Markov word model per word in the vocabulary, there is only one language model per vocabulary. In the case of a 3-gram language model, the probabilities are collected for all 3-uples of words that appear in a large body of text written in the language considered. Thus, and by example, the probability of the 3-uple "the phone rings" is significantly higher than the probability of the 3-uple "the phone drinks". It should be noted that this difference in probabilities is independent of the corresponding acoustic signals. Therefore, a language model may be determined once for a given language, whereas Markov word models improve as they are trained on a speaker-by-speaker basis. By the same token, the same language model could also be used for AHR. However, for AHR a simple dictionary look-up is preferred since it can be accomplished readily by inspection.
The handwriting model trainer 28 preferably operates in accordance with an elastic stroke matching technique disclosed by C. C. Tappert in an article entitled "Adaptive On-Line Handwriting Recognition", IEEE International Conference on Pattern Recognition Proceedings, (1984). As employed herein elastic matching clusters elastically deformed input strokes, as represented by their associated stroke vectors, into stroke templates to form character prototypes.
Coupled to the output of each of the models 32 and 34 is a Likelihood Estimator (LE), specifically, a handwriting LE 36 and a speech LE 38. The general purpose of each of the LEs, in conjunction with their associated word models, is to assign probabilities to strings of words in response to input speech or handwriting signals. Each LE is aided by a Language Model (LM), specifically a handwriting LM 36 a and a speech LM 38 a, whose role is to reduce the search space by a priori discarding unlikely strings of words or characters. By example, and as was previously noted, the string of words "the phone drinks" has a much smaller likelihood or probability of occurrence than the string "the phone rings". It would thus be inefficient to carry out a full likelihood computation on the former string as opposed to the latter string. The LM may be of a statistical nature, such as one based on N-gram probabilities, or as simple as a machine readable dictionary. Furthermore, the LM for speech may be different than the LM for handwriting. In a presently preferred embodiment of the invention the speech LM 38 a is a statistical 3-gram model and the handwriting LM 36 a is a dictionary look-up.
After training, and during decoding of on-line speech and/or handwriting, the LEs 36 and 38 receive the feature vectors from blocks 24 and 26, respectively, and, in conjunction with the associated word models, and in some cases the associated LM scores, generate estimations of probable matching words. The speech estimator 38 outputs a list (L1) of word estimations and the handwriting estimator also outputs a list (L2) of word estimations. By example, for handwriting the input strokes are compared to prototypes via elastic matching and identified characters are grouped and compared to a dictionary of valid character sequences to derive the list of probable word candidates (L2). For speech, the likelihood of each Markov word model producing the input utterance is computed, after pruning by the 3-gram LM 38 a, and those words resulting in the highest scores are placed on the list of word candidates (L1). In both cases (speech and handwriting), the size of the list is preferably statically set using a predetermined reasonable maximum size, such as 15, or may be set dynamically using a threshold. In the latter case, all words whose likelihoods fall within the selected threshold are maintained on the list, and the other words discarded.
Each likelihood score in L1 and L2 is applied to a scaling block 40 to ensure that they are in a comparable range. This is typically necessary in that speech and handwriting likelihoods are obtained independently. This scaling can be implemented, by example, by normalizing each likelihood score in L1 and L2 by the average likelihood score in L2 and L2, respectively. In accordance with the example provided above for the string "the phone rings" the following list may be produced.
The scaled likelihood scores in both L1 and L2 (forming lists SL1 and SL2, respectively) are applied to a Likelihood Merging block (LM) 42. For any sl1 and sl2 corresponding to the same word in SL1 and SL2, respectively, a merged likelihood (l) is computed as:
where the function f(.) is selected in accordance with the intended application environment. In a presently preferred embodiment f(.) is given by:
where α+β=1.
Assuming for the following example that N≠M and the use of dynamic lists during training, then:
Scaling provides the scaled lists SL1 and SL2 as:
Assuming that sl1 p and sl2 q correspond to some word (1≦p≦N, 1≦q≦M), then the total likelihood for this word is:
As such, there is also shown above one suitable method for determining the weights 37 during training. In an ideal case, there is one pair (α, β) for each word in the vocabulary.
In the foregoing example, α and β may be the weights 37 or some overriding values (WOVR) selected by the user.
The output of the LM block 42 is a list (L) of possible words corresponding to the acoustic and/or stroke inputs. The list of words is ordered by probability. In general, the members of the list L are a function of SL1, SL2, and the associated weights α and β (W). The list of possible words is applied to a decoder 44 that selects the candidate word having the highest probability. This selected candidate word is considered to be a recognized message and becomes available to use by other applications including the user interface 12. These other applications may include, by example, a word processing program.
It is within the scope of the invention to provide automatic pruning or trimming of the scaled lists SL1 and SL2 such that L contains only those candidate words that are common to both SL1 and SL2. This method of trimming is readily accomplished by inspection. An initial trimming of the two word lists simplifies the task of decoder 42 in that fewer candidate words need be considered for likelihood merging.
As was previously noted it is also within the scope of the invention to provide a manual or automatic weight override (WOVR) to the LM block 42. The WOVR enables a user to vary the weights to provide a desired value. This may be accomplished, for example, if the user is inputting speech information that included words considered to be of great difficulty for the speech recognizer to process. The weights associated with these words are manually changed to give greater weight to the handwriting likelihood scores as opposed to the speech likelihood scores. For example, the weight associated with the words selected from the handwriting recognizer may be set to β=1.0 and the weight associated with the speech selected words set to α=0. This is equivalent to disregarding speech input altogether. Such weight override may also be used in an error correction mode where sequential activation of speech and handwriting is desired. For example, a word known to have a number of easily confusable characters could be entered via the speech transducer 22 as opposed to the handwriting transducer 20, as long as α=1.0 and β=0.
Referring now to FIG. 2 there is illustrated a second embodiment of a message recognition system 50. In FIG. 2 components that are common with FIG. 1 have the same reference designators and operate in the manner described above.
The embodiment of FIG. 2 operates with a feature vector that is a combination of FVS and FVH. The speech and handwriting feature vectors are each generated in response to simultaneously provided acoustic and handwriting input. In that the sampling rates and durations of the input are different for the speech and handwriting channels a normalization block 52 normalizes the two inputs to a common time base and combines the normalized feature vectors into a single feature vector. This combination may be a synthetic concatenation of the four dimensional handwriting vector with the 20 dimensional speech vector, as indicated in FIG. 4e, to provide a 24 dimensional combined vector (FVS,H). Other techniques may also be employed for combining the two feature vectors. For example, the feature vectors may be combined by taking a product of the two underlying separate vector spaces, thereby providing an 80-dimensional combined feature vector. Other, more complex, combinatorial techniques may also be employed.
It is noted that the above illustrates various examples only, and that other dimensions could be considered. A general consideration is that dimFVH<dimFVS.
For example, it is within the scope of the invention to treat the input handwritten strokes in the frequency domain, as opposed to a conventional spatial treatment, and to Fourier transform the stroke information. A four or more dimensional stroke feature vector thus represents frequency band amplitude (a) information that is combined with the speech feature vector, as indicated in FIG. 4f. It is pointed out that the frequency domain processing for handwriting may also be employed with the system illustrated in FIG. 1.
In any case, the combined feature vectors are applied during training to a speech and handwriting model trainer processor 54 for training appropriate word models 56. By example, each word model 56 may be a hidden Markov model. It is noted that, due to the combined feature vectors, the states and traversal of the states of the word models 56 will differ from those of the Markov word models employed for only speech, as in FIG. 1. However, once structurally specified, the parameters of these models are trained in a conventional fashion as described, for example, in the above referenced journal article entitled "A Maximum Likelihood Approach to Continuous Speech Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-5, No. 2, March 1983 pp. 179-190 by L. Bahl et al. One difference is that, for example, the output distributions in the word models block 56 are joint distributions (i.e., probabilities of joint phoneme/word fragment pairs) reflecting both the speech and handwriting inputs.
The word models 56 are utilized by a Likelihood Estimator (LE) 58 that generates a list (L) of the most probable words based on the combined handwriting and speech input in accordance with a Language Model (LM) 58 a. Preferably, for the joint use of speech and handwriting information the LM 58 a is based on the 3-gram language model, in that a dictionary look-up model is generally inappropriate for speech processing.
The decoder 44 selects the element of the list having the highest probability and, thus, generates the recognized message.
Referring to FIG. 5 there is illustrated in flow chart form the operation of the systems 10 and 50. A generalized discussion of FIG. 5 is first provided followed by a detailed description of the operation of each of the blocks shown therein.
At block 60 there occurs data acquisition of pen strokes and/or acoustic (speech) information. Acquired strokes and acoustic data are operated on by the FV processors 24 and 26, respectively. These two tasks may be performed simultaneously or at different times, depending upon how the handwriting information and the speech information are intended to be integrated.
During a training mode of operation (block 62 ) the acquired information is analyzed, in reference to a known, training set of handwritten and spoken characters and words, to train the underlying models.
During use, the training block 62 is bypassed and the feature vectors are directed instead to the likelihood estimators (block 64). The likelihood estimators employ the feature vectors to generate candidate words from assemblages of characters and/or phonemes. The decoder selects the most probable word based upon the weights and other relevant parameters.
Recognized messages are thereafter made available for use (block 66). By example, a recognized message may be simply converted to an alphanumeric format and displayed upon the display 14. The recognized message may also be provided to any application that would conventionally receive messages from a keyboard such as, by example, a word processing program.
In accordance with the invention there is provided an integrated use of ASR and AHR. By integrated use it is intended that (a) ASR and AHR are used simultaneously, and a decoding decision is based upon an overall likelihood score, either joint or merged, that an assemblage of characters or phonemes represents a specific word; or that (b) ASR and AHR are used sequentially, but with one being used to correct any errors generated by the other. It should be noted that ASR and AHR need not both be employed by the same user, so long as both process a consistent message. In some applications, it may be desirable to have ASR operate with one user while AHR operates with another.
In regard to the simultaneous use of acoustic and handwriting information it is noted that both contribute to an overall decoding decision. Ambiguous characters in the handwritten information are recognized primarily on the basis of the corresponding spoken utterance and, conversely, the acoustically confusable words are recognized primarily on the basis of the handwriting information. For this technique words are uttered at the same time as they are written and, as a result, natural pauses appear in the spoken utterance. Hence, it is most advantageous to integrate isolated word speech recognition with mixed cursive and discrete handwriting recognition. If, however, the environment is too noisy for this technique it may be more appropriate to replace the traditional ASR by an automatic lip reading recognizer of the type referred to above.
In regard to the sequential use of AHR and ASR, it is noted that AHR may be employed as an enhancement to ASR. For this approach, ASR is considered to be the primary recognition means and AHR is employed primarily as a natural and convenient interface to correct ASR errors that are displayed either on a transparent electronic tablet or on the display terminal 14. This technique is also suitable for entering new words, such as proper nouns, which are not in the ASR vocabulary but present in an AHR dictionary.
Alternatively, ASR may be employed as an enhancement to AHR. For this approach, AHR is considered the primary recognition means and ASR is used for the purpose of correcting AHR errors. By pointing on AHR errors in some pre-defined manner the speech transducer 22 is activated so that the user may pronounce a correct version of the wrongly decoded handwritten word.
For all of the above combinations of AHR and ASR, the integrated use of ASR and AHR has been found to be advantageous for several reasons.
Firstly, there exists complementarity in information. That is, information gathered through speaking and information gathered through handwriting complement one another. For example, the letters U and V are often difficult to recognize using AHR but pose no problem to ASR. Conversely, the so-called "e-set" B, C, D, E, G, P, T, V, Z is notoriously difficult to recognize using ASR but does not constitute as difficult a task for AHR.
Secondly, there is provided complementarity in performance. That is, the manner in which speech and handwriting recognition algorithms operate often complement one another. For example, in AHR the probability of making a word error increases as the length of the word increases, because more characters may be misrecognized. However, in ASR this probability decreases as the length of the word increases, because there are fewer words that are confusable.
Thirdly, there is provided complementarity in functionality. That is, those functions most readily implemented using ASR and AHR complement each other. For example, AHR exhibits editing capabilities which are more difficult to implement using ASR, such as deleting a word by using the writing implement to cross out the word, or indicating that words or paragraphs are to be interchanged. On the other hand some functions, such as generating a large body of text, may be implemented more efficiently with the speech transducer 22 than through the use of the writing transducer 20.
As was previously stated, the integrated use of speech and handwriting recognition results from the Integrated Speech and Writing Interface (ISWI) 18. Word recognition using ISWI may be accomplished under twelve possible conditions, depending on whether speech is isolated or continuous, whether handwriting is discrete or cursive, and whether the two recognition procedures are sequential, merged or joint. In this regard, the sequential use of AHR and ASR may be best employed for error correction, the joint use of AHR and ASR is achieved through the use of joint likelihood estimation, and the merged use of AHR and ASR employs weighted likelihood estimations of the form (αL1+βL2).
Furthermore, the implementation of ISWI depends on three processes: (a) data acquisition (block 60); (b) training, assuming a user-dependent mode (block 62); and (c) decoding (block 64). These processes are now discussed in greater detail.
Data Acquisition (Block 60)
There are two most prevalent modes of acquiring data for the use of the message recognition systems 10 and 50. In a training mode, except for the joint likelihood mode, a user produces stroke and acoustic information separately by writing and speaking generally different training materials. For example, a user may train ASR on 100 or 200 phonetically balanced sentences, and train AHR on a few sentences comprising several instances of all letters of the alphabet. In a decoding mode, except for the sequential use mode, a user produces stroke and acoustic information simultaneously by writing and speaking the same message. This procedure is especially suitable for cursive writing and isolated speech in that, since writing is slower than speaking, pauses naturally appear between each spoken word.
Training (Block 62)
The training process may be partitioned into two parts. First, suitable feature vectors are generated for both speech and handwriting. Secondly, each feature vector is used in conjunction with some class of models, such as hidden Markov models, and the associated word model parameters are trained accordingly.
If training is done separately for speech and handwriting the selected feature vectors can be those ordinarily used in either domain. This results in the generation of pairs of feature vectors that correspond to the acoustic and stroke information, respectively. Such pairs of feature vectors may be utilized in two distinct ways. A simplest method uses the vectors as in FIG. 1 in two separate classes of models, one for speech and the other for handwriting. The other method, embodied in the system of FIG. 2, normalizes the input data, combines features vectors, and trains extended word models that incorporate both acoustic and stroke information.
As was stated, known methods employed for speech and handwriting recognition to generate and train models, such as hidden Markov models, may be employed for the integration of these two techniques. The use of neural networks as models for speech and handwriting is also within the scope of the invention. These methods, in general, all operate on abstract mathematical objects, such as strings of labels or points in vector spaces. However, in the case of joint training, the underlying model structures, such as word templates or neural network topologies, are different for ISWI than those used for AHR or ASR alone.
The following articles describe suitable embodiments of neural networks for speech recognition: A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K. Lang, "Phoneme Recognition Using Time-Delay Neural Networks", IEEE Trans. on Acoustic, Speech and Signal Processing, Vol. 37, No. 3, pp. 328-339, March 1989; and S. G. Kong and B. Kosko, "Differential Competitive Learning for Centroid Estimation and Phoneme Recognition", IEEE Trans. Neural Networks, Vol. 2, No. 1, pp. 118-124, January 1991.
The following articles describe suitable embodiments of neural networks for handwriting recognition: P. Morasso, "Neural Models of Cursive Script Handwriting", Proc. 1989 International Joint Conference on Neural Networks, Volume II, pp. 539-542, Washington, D.C., June 1989; and I. Guyon, P. Albrecht, Y. Le Cun, J. Denker, and W. Hubbart, "Design of a Neural Network Character Recognizer for a Touch Terminal", Pattern Recognition, Volume 24, 1991, in press.
Decoding (Block 64)
Just as separate word model training and joint word model training are distinguished, a same methodological dichotomy exists for decoding.
For the case where acoustic and stroke information separately contribute to an overall decoding decision, one suitable general decision making procedure is based on multiple expert opinions, as in consensus theory. For example, the overall likelihood score may be a weighted sum of the likelihood scores corresponding to each source of information, the weights (W) being selected during training on the basis of such external factors as the length of the candidate word or the presence of noise.
The following three journal articles are referenced as teaching decision making strategies suitable for use by the decision decoder 44 in selecting between acoustic and stroke information: "Combining Probability Distributions from Dependent Information Sources", Management Science, Vol. 27, No. 4, April 1981, pages 479-488 (R. L. Winkler); "On the Aggregation of Individual Probability Estimates", Management Science, Vol. 27, No. 8, August 1981, pages 959-964 (R. F. Bordley et al.); and "Consensus and Evidence", Pattern Recognition in Practice II, Elsevier Science Publishers B.V. (North Holland), 1986, pages 523-546 (C. Berenstein et al.)
Alternatively, when both speech and handwriting information simultaneously contribute to the decoding decision, as in the message recognition system 50 of FIG. 2, the decision is made on the basis of the LE 58 operating on the extended word models 56 in accordance with the language model 58 a.
ISWI has application in a number of fields, including the field of human factor experiments. More generally, ISWI is well suited in any domain where the complementarity of speech and handwriting information can be exploited, and, in particular, for error-correcting and editing purposes.
Two series of experiments were performed. A purpose of the first series was to obtain information regarding the complementarity of speech and handwritten information. A purpose of the second series was to obtain information regarding the complementarity of speech and handwriting recognition performance.
In the first series of experiments a vocabulary of 24 words composed of eight colors, eight objects, and eight prepositions was employed. A subject (not previously familiar with AHR) wrote in writer-independent mode each of the 24 words in a discrete manner. Six words were incorrectly decoded, yielding a handwriting error rate of 25%. The same words were spoken to ASR in an isolated utterance manner and decoded in a speaker-dependent mode. Twelve words were incorrectly decoded, yielding a speech error rate of 50% on this relatively difficult vocabulary. An intersection between the handwriting-ambiguous and speech-ambiguous words was found to reduce to three words, one of which was outside the vocabulary of the speech recognizer. The combination thus yields an overall error rate of only 8%.
It is noted that in the case of AHR all of the errors (such as "ORANGE" decoded as "ORAIVGE") could readily be detected and corrected by automatic reference to an underlying dictionary. This post-processing may be implemented through a simple algorithm which, for a given sequence of characters, determines a word from the given vocabulary that best fits this sequence. The algorithm need only compute the number of matching letters in the input sequence and each tentative word. In the case of ASR, however, it was discovered that this post-processing technique is capable of only limited corrections.
It is also noted that there occurred an improvement of handwriting recognition with time. After observing the errors produced during the first trial the subject was able to obtain an improved accuracy during a second trial. A similar improvement, although more limited, was observed when using ASR.
In the second series of experiments, another subject produced the sentence "Two articles did not describe speech perception by eye and ear" in three different ways: (a) by writing the sentence on AHR in writer-dependent, run-on mode; (b) by speaking the sentence to ASR in speaker-dependent, isolated utterance mode; and (c) by writing short words (two, did, not, by, eye, and, ear) on AHR and speaking long words (articles, describe, speech, perception) to ASR in the same conditions as in (a) and (b), respectively. Condition (a) produced two errors, both of them made on long words ("articles" and "perception"). Condition (b) produced four errors, three on short words ("two, by, eye") and one on the long word "articles". However, condition (c) produced only one error on the long word "articles", which was misrecognized by ASR.
These experiments provide explicit evidence that the use of two sources of information, specifically handwriting and speech, greatly enhances the word recognition process.
The teaching of the invention may be employed to provide a superior "human factor workbench" that addresses the problems noted above in regard to human factors experimental techniques. Within the controlled environments typical of all psychological experiments the invention is amenable to several implementations. For example, for short experiments involving many different users (for which training would be justifiable), ISWI provides for a writer/speaker-independent system relying for overall accuracy on the constraints imposed on vocabularies and grammars. For long experiments involving the same set of subjects over a sustained period of time, ISWI provides a writer/speaker-dependent system with capabilities of connected speech and/or cursive writing recognition in order to enhance the naturalness of the interface.
While the invention has been particularly shown and described with respect to preferred embodiments thereof, it will be understood by those skilled in the art that changes in form and details may be made therein without departing from the scope and spirit of the invention.
|Cited Patent
|Filing date
|Publication date
|30 Jul 1975
|13 Jul 1976
|International Business Machines Corporation
|Regional context maximum likelihood error correction for OCR, keyboard, and the like
|31 Dec 1975
|17 May 1977
|International Business Machines Corporation
|Segmentation mechanism for cursive script character recognition systems
|24 Jan 1983
|17 Mar 1987
|Tokyo Shibaura Denki Kabushiki Kaisha
|Pattern recognition apparatus and method for making same
|9 Oct 1986
|23 Feb 1988
|International Business Machines Corporation
|System for automatic adjustment and editing of handwritten text image
|3 Nov 1986
|15 Mar 1988
|International Business Machines Corporation
|Recognition system for run-on handwritten characters
|26 Aug 1985
|5 Apr 1988
|Gersh Korsinsky
|Video computer
|15 Oct 1985
|28 Jun 1988
|The Palantir Corporation
|Means for resolving ambiguities in text based upon character context
|24 Sep 1986
|27 Sep 1988
|Buckley Bruce S
|Self-organizing circuits
|6 Nov 1986
|14 Feb 1989
|The Research Foundation Of The State University Of New York
|Pattern recognition method and apparatus
|27 Jul 1988
|15 Aug 1989
|The United States Of America As Represented By The Secretary Of The Navy
|Intelligent security assessment system
|11 Mar 1988
|6 Mar 1990
|Kabushiki Kashia Toshiba
|Intelligent work station
|27 Nov 1989
|12 Feb 1991
|Motorola, Inc.
|Unforgeable personal identification system
|18 Aug 1989
|28 Feb 1990
|Fujitsu Limited
|A pattern recognition apparatus and method for doing the same
|Title not available
|Title not available
|Title not available
|Title not available
|Title not available
|Title not available
|23 Sep 1991
|2 Apr 1992
|Roth Richard G
|Audio-augmented handwriting recognition
|"A Handwriting Recognition System Based on the Properties and Architectures of the Human Motor System" L. Schomaker et al.
|"Adaptive Labeling: Normalization of Speech by Adaptive Transformations Based on Vector Quantization", A. Nadas, et al. Proc. 1988 Int. Conf. on Acou. Speech, Signal Proc.
|"Adaptive On-Line Handwriting Recognition", IEEE Seventh International Conference on Pattern Recognition Proceedings, Jul. 30-Aug. 12, 1984, by C. C. Tappert.
|"Application of an Auditory Model to Speech Recognition", J. Acoust, Soc. Am. 85(6) Jun. 1989, J. Cohen.
|"Automatic Lipreading to Enhance Speech Recogniton" CH2145-1, IEEE 1985., D. Petajan.
|"Combining Probability Distributions from Dependent Information Sources" Management Science, vol. 27, No. 4, Apr. 1981, R. Winkler.
|"Consensus and Evidence" Pattern Recognition Practice II, 1986, pp. 523-546, C. Berenstein et al.
|"Design of a Neural Network Character Recognizer for a Touch Terminal" Pattern Recognition, vol. 24, 1991, in press.
|"Differential Competitive Learning for Centroid Estimation and Phoneme Recognition", S. Kong et al. IEEE Transactions on Neural Networks, vol. 2, No. 1, Jan. 1991.
|"Experiments on Neural Net Recognition of Spoken and Written Text" IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 36, No. 7 Jul. 1988 D. Burr.
|"Flexible, Space-Time Segmenter for Handwritten Words" IBM Technical Disclosure Bulletin, vol. 29, No. 11, Apr. 1987, A. S. Fox et al.
|"Handwriting Recognition by SUbstroke Segmentation and Composition" IBM Technical Discl. Bulletin, vol. 27, No. 9, Feb. 1985, J. Kurtzberg et al.
|"Handwriting/Speech Signal Analyzer" IBM Technical Disclosure Bulletin, vol. 22, No. 5 Dec. 1989, J. Gould et al.
|"Neural Models of Cursive Script Handwriting", P. Morasso, Proc. 1989 International Joint Conf. on Neural Networks, vol. II, pp. 539-542, Jun. 1989.
|"On the Aggregation of Individual Probability Estimates" Management Science, vol. 27, No. 8, Aug. 1981, R. Bordley et al.
|"Phoneme Recognition Using Time-Delay Neural Networks", A. Waibel et al. IEEE Trans. on Acoustics, Speech and Signal Processing, vol. 37. No. 3, Mar. 1989.
|"Segmentation Procedure for Handwritten Symbols and Words" IBM Technical Discl. Bulletin vol. 25, No. 7B, Dec. 1982, J. Kurtzberg et al.
|"Speech Recognition Using Noise-Adaptive Prototypes" A. Nadas, reprinted from IEEE Transactions on Acoustics, Speech and Signal Processing vol. 37, No. 10, 1989.
|Patent Abstract of Japan, vol. 10, No. 45 (P-430) Feb. 21, 1986 & JP, A, 60 189 070 (Fujitsu K.K. ) Sep. 26, 1985.
|Patent Abstract of Japan, vol. 11, No. 87 (P-557) Mar. 17, 1987 & JP,A, 61 240 361 (Hitachi Electronics Eng. Co. Ltd.) Oct. 25, 1986.
|Patent Abstract of Japan, vol. 6, No. 137 (P-130) Jul. 24, 1982, & JP, A, 57 059 241 (Sasaki Nobuhiko) Apr. 1982.
|Patent Abstract of Japan, vol. 7, No. 146 (P-206) Jun. 25, 1983 & JP,A,58 058 637 (Nippon Denki K.K.) Apr. 7, 1983.
|Patent Abstract of Japan, vol. 7, No. 249 (P-234) Nov. 5, 1983 & JP, A, 58 134 371 (Nippon Denki K.K.) Aug. 10, 1983.
|Citing Patent
|Filing date
|Publication date
|5 Jan 2000
|25 Jan 2005
|Apple Computer, Inc.
|Universal interface for retrieval of information in a computer system
|28 Jan 2002
|7 Jun 2005
|Edwin A. Suominen
|Message recognition using shared language model
|20 Apr 2001
|16 Aug 2005
|International Business Machines Corporation
|Decision making in classification problems
|13 May 2002
|10 Jan 2006
|Microsoft Corporation
|Correction widget
|25 Apr 2000
|25 Apr 2006
|Microsoft Corporation
|Language model sharing
|30 Jul 2002
|14 Nov 2006
|Microsoft Corporation
|Correcting recognition results associated with user input
|8 Sep 2005
|28 Aug 2007
|Canon Kabushiki Kaisha
|Information processing apparatus and method, computer-readable memory, and program
|27 Jul 2005
|28 Aug 2007
|Microsoft Corporation
|Correction widget
|18 Aug 2003
|9 Sep 2008
|Acer Incorporated
|Recognition method and the same system of ingegrating vocal input and handwriting input
|12 Aug 2005
|16 Dec 2008
|Silverbrook Research Pty Ltd
|Device and method for communication
|15 Dec 2003
|17 Mar 2009
|Microsoft Corporation
|Multi-modal handwriting recognition correction
|17 Feb 2004
|28 Apr 2009
|Silverbrook Research Pty Ltd
|Line extraction in digital ink
|27 Jul 2005
|14 Jul 2009
|Microsoft Corporation
|Correction widget
|18 Sep 2007
|4 Aug 2009
|Microsoft Corporation
|Speech recognition using a state-and-transition based binary speech grammar with a last transition value
|9 Dec 2003
|25 Aug 2009
|Microsoft Corporation
|Accuracy model for recognition signal processing engines
|21 Nov 2007
|8 Dec 2009
|Silverbrook Research Pty Ltd
|Method and system for line extraction in digital ink
|1 Dec 2004
|8 Dec 2009
|Tellme Networks, Inc.
|Method and system for selecting grammars based on geographic information associated with a caller
|6 Apr 2006
|26 Jan 2010
|Natural language query system for accessing an information system
|6 Jul 2006
|9 Mar 2010
|Hewlett-Packard Development Company, L.P.
|Data input apparatus and method
|3 Dec 2003
|7 Dec 2010
|Microsoft Corporation
|Scaled text replacement of ink
|31 Mar 2006
|22 Feb 2011
|Microsoft Corporation
|Language model sharing
|23 Nov 2008
|19 Apr 2011
|Silverbrook Research Pty Ltd
|System for electronic transmission of handwritten information
|26 Feb 2008
|13 Dec 2011
|Microsoft Corporation
|Handwriting symbol recognition accuracy using speech input
|1 Dec 2004
|27 Dec 2011
|Apple Inc.
|Universal interface for retrieval of information in a computer system
|22 Mar 2005
|19 Jun 2012
|Optical Research Partners Llc
|Message recognition using shared language model
|1 Dec 2009
|14 May 2013
|Hyundai Motor Company
|Combined lip reading and voice recognition multimodal interface system
|27 Sep 2010
|6 May 2014
|Apple Inc.
|Electronic device with text error correction based on voice recognition data
|21 Dec 2012
|18 Nov 2014
|Apple Inc.
|Service orchestration for intelligent automated assistant
|21 Dec 2012
|2 Dec 2014
|Apple Inc.
|Personalized vocabulary for digital assistant
|4 Mar 2013
|6 Jan 2015
|Apple Inc.
|Paraphrasing of user requests and results by automated digital assistant
|21 Dec 2012
|27 Jan 2015
|Apple Inc.
|Determining user intent based on ontologies of domains
|15 Oct 2012
|5 May 2015
|Nuance Communications, Inc.
|Text/character input system, such as for use with touch screens on mobile phones
|22 Jul 2013
|7 Jul 2015
|Apple Inc.
|Electronic device with text error correction based on voice recognition data
|21 Dec 2012
|25 Aug 2015
|Apple Inc.
|Using event alert text as input to an automated assistant
|4 Mar 2014
|17 Nov 2015
|Apple Inc.
|User profiling for voice input processing
|21 Jul 2009
|5 Jan 2016
|The Quantum Group, Inc.
|System and method enabling bi-translation for improved prescription accuracy
|21 Mar 2011
|16 Feb 2016
|Apple Inc.
|Device access using voice authentication
|13 Jun 2014
|29 Mar 2016
|Apple Inc.
|System and method for emergency calls initiated by voice command
|10 Jan 2011
|19 Apr 2016
|Apple Inc.
|Intelligent automated assistant
|2 Apr 2008
|3 May 2016
|Apple Inc.
|Methods and apparatus for altering audio output signals
|26 Sep 2014
|10 May 2016
|Apple Inc.
|Intelligent automated assistant for TV user interactions
|6 Mar 2014
|14 Jun 2016
|Apple Inc.
|Context-sensitive handling of interruptions
|30 Sep 2014
|30 Aug 2016
|Apple Inc.
|Exemplar-based natural language processing
|17 Aug 2006
|11 Oct 2016
|Microsoft Technology Licensing, Llc
|Operating system program launch menu search
|6 Mar 2012
|1 Nov 2016
|Apple Inc.
|Handling speech synthesis of content for multiple languages
|12 Mar 2013
|15 Nov 2016
|Apple Inc.
|Device, method, and user interface for voice-activated navigation and browsing of a document
|23 Sep 2014
|22 Nov 2016
|Apple Inc.
|Method for supporting dynamic grammars in WFST-based ASR
|13 Nov 2014
|6 Dec 2016
|At&T Intellectual Property Ii, L.P.
|Method and system for automatically detecting morphemes in a task classification system using lattices
|17 Jun 2015
|3 Jan 2017
|Apple Inc.
|Mobile device having human language translation capability with positional feedback
|9 Jun 2012
|17 Jan 2017
|Apple Inc.
|Intelligent automated assistant
|9 Sep 2013
|21 Feb 2017
|Apple Inc.
|Context-sensitive handling of interruptions by intelligent digital assistant
|6 Jun 2014
|28 Feb 2017
|Apple Inc.
|Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
|6 Jun 2014
|11 Apr 2017
|Apple Inc.
|System and method for user-specified pronunciation of words for speech synthesis and recognition
|29 Sep 2014
|11 Apr 2017
|Apple Inc.
|Analyzing audio input for efficient speech and music recognition
|4 Apr 2016
|18 Apr 2017
|Apple Inc.
|Intelligent text-to-speech conversion
|29 Sep 2014
|25 Apr 2017
|Apple Inc.
|Better resolution when referencing to concepts
|13 Nov 2015
|25 Apr 2017
|Apple Inc.
|User profiling for voice input processing
|5 Jun 2014
|25 Apr 2017
|Apple Inc.
|System and method for detecting errors in interactions with a voice-based digital assistant
|25 Aug 2015
|9 May 2017
|Apple Inc.
|Caching apparatus for serving phonetic pronunciations
|21 Dec 2015
|9 May 2017
|Apple Inc.
|Fast, language-independent method for user authentication by voice
|30 Mar 2016
|30 May 2017
|Apple Inc.
|Intelligent automated assistant for TV user interactions
|25 Aug 2015
|30 May 2017
|Apple Inc.
|Social reminders
|7 Dec 2015
|4 Jul 2017
|Apple Inc.
|Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
|28 Apr 2014
|4 Jul 2017
|Apple Inc.
|System and method for updating an adaptive speech recognition model
|12 Dec 2014
|18 Jul 2017
|Apple Inc.
|Disambiguating heteronyms in speech synthesis
|30 Sep 2014
|25 Jul 2017
|Apple Inc.
|Reducing the need for manual start/end-pointing and trigger phrases
|31 Aug 2015
|1 Aug 2017
|Apple Inc.
|Competing devices responding to voice triggers
|18 Sep 2014
|15 Aug 2017
|Apple Inc.
|Determining domain salience ranking from ambiguous words in natural speech
|22 May 2015
|12 Sep 2017
|Apple Inc.
|Predictive text input
|28 May 2015
|10 Oct 2017
|Apple Inc.
|Text prediction using combined word N-gram and unigram language models
|25 Feb 2015
|24 Oct 2017
|Apple Inc.
|Text correction processing
|20 Apr 2001
|21 Nov 2002
|International Business Machines Corporation
|Decision making in classification problems
|28 Jan 2002
|20 Mar 2003
|Suominen Edwin A.
|Text processing system
|14 Dec 2001
|19 Jun 2003
|Koninklijke Philips Electronics N.V.
|Input of data using a combination of data input systems
|9 Apr 2002
|9 Oct 2003
|Microsoft Corporation
|Assignment and use of confidence levels for recognized text
|13 May 2002
|13 Nov 2003
|Microsoft Corporation
|Correction widget
|17 Jun 2002
|18 Dec 2003
|Microsoft Corporation
|Integration of speech and stylus input to provide an efficient natural input experience
|30 Jul 2002
|5 Feb 2004
|Microsoft Corporation
|Correcting recognition results associated with user input
|17 Feb 2004
|26 Aug 2004
|Dimitrios Koubaroulis
|Line extraction in digital ink
|18 Aug 2003
|11 Nov 2004
|Chao-Shih Huang
|Recognition method and the same system of ingegrating vocal input and handwriting input
|1 Dec 2004
|14 Apr 2005
|Apple Computer, Inc. One Infinite Loop
|Universal interface for retrieval of information in a computer system
|20 Feb 2004
|26 May 2005
|Acer Inc.
|Voice interactive method and system
|8 Nov 2004
|9 Jun 2005
|Myers Gregory K.
|Method and apparatus for fusion of recognition results from multiple types of data sources
|9 Dec 2003
|9 Jun 2005
|Microsoft Corporation
|Accuracy model for recognition signal processing engines
|15 Dec 2003
|16 Jun 2005
|Microsoft Corporation
|Multi-modal handwriting recognition correction
|3 Dec 2003
|23 Jun 2005
|Microsoft Corporation
|Scaled text replacement of ink
|22 Mar 2005
|4 Aug 2005
|Suominen Edwin A.
|Message recognition using shared language model
|12 Aug 2005
|8 Dec 2005
|Silverbrook Research Pty Ltd
|Device and method for communication
|8 Sep 2005
|5 Jan 2006
|Canon Kabushiki Kaisha
|Information processing apparatus and method, computer-readable memory, and program
|30 Jan 2003
|27 Jul 2006
|Eric Thelen
|Method and device for the rapid, pattern-recognition-supported transcription of spoken and written utterances
|31 Mar 2006
|3 Aug 2006
|Microsoft Corporation
|Language model sharing
|6 Apr 2006
|17 Aug 2006
|Natural language query system for accessing an information system
|6 Jul 2006
|25 Jan 2007
|Hewlett-Packard Development Company, L.P.
|Data input apparatus and method
|20 Aug 2004
|23 Aug 2007
|Mindlin Bernardo G
|Topological voiceprints for speaker identification
|18 Sep 2007
|21 Feb 2008
|Microsoft Corporation
|Compact Easily Parseable Binary Format for a Context-Free Grammar
|21 Nov 2007
|11 Dec 2008
|Silverbrook Research Pty Ltd
|Method and system for line extraction in digital ink
|23 Nov 2008
|2 Apr 2009
|Silverbrook Research Pty Ltd
|System for electronic transmission of handwritten information
|26 Feb 2008
|27 Aug 2009
|Microsoft Corporation
|Handwriting symbol recognition accuracy using speech input
|13 Apr 2009
|24 Sep 2009
|Silverbrook Research Pty Ltd
|System For Line Extraction In Digital Ink
|21 Jul 2009
|28 Jan 2010
|The Quantum Group, Inc.
|System and method enabling bi-translation for improved prescription accuracy
|1 Dec 2009
|24 Mar 2011
|Hyundai Motor Company
|Combined lip reading and voice recognition multimodal interface system
|27 Sep 2010
|29 Mar 2012
|Wagner Oliver P
|Electronic device with text error correction based on voice recognition data
|29 Dec 2011
|5 Jul 2012
|Samsung Electronics Co., Ltd.
|System and method for exchange of scribble data between gsm devices along with voice
|15 Oct 2012
|17 Apr 2014
|Nuance Communications, Inc.
|Text/character input system, such as for use with touch screens on mobile phones
|16 Dec 2013
|3 Jul 2014
|Kabushiki Kaisha Toshiba
|Shaping device and shaping method
|13 Nov 2014
|12 Mar 2015
|At&T Intellectual Property Ii, L.P.
|Method and system for automatically detecting morphemes in a task classification system using lattices
|10 May 2013
|13 Nov 2014
|Audi Ag
|Kraftfahrzeug-Eingabevorrichtung mit Zeichenerkennung
|U.S. Classification
|International Classification
|Cooperative Classification
|European Classification
|15 Dec 2004
|Fee payment
Year of fee payment: 4
|16 Mar 2009
|Maintenance fee reminder mailed
|4 Sep 2009
|Lapse for failure to pay maintenance fees
|27 Oct 2009
|Expired due to failure to pay maintenance fee
Effective date: 20090904


text:
Green company
Ballan is also cutting edge
in their business approach
Ballan SpA is not only the company that has contributed most to innovation in the key areas of its sector: mechanics, electronic, materials, design and, naturally, safety.
Aware of its own social responsibility, Ballan SpA is now a green company implementing environmental protection and energy saving policies.
A company that invests significant resources in Research and Development, personnel training, and improvements to production and storage systems. A company that has banned all pollutant substances and materials from its production process.
To look ahead to the future with a clear conscience.

subset: commoncrawl


text:
GraphicRiver Ideas Generator Logo 5687571
Logo Template \ Humans
Change \ Elements \ Beautiful \ Flyers \ Different \ Textures \ Backgrounds
Logo Template - Ideas Generator Logo 5687571 by Kapacyko @ GraphicRiver
Logo Template \ Humans
Fully Editable Logo, AI, EPS, CDR, PNG files
Used free font link in the zip folder
Easy work and good luck
Don't forget to rate if you like it!
You bought a logo, but You cannot configure it the way you would like? I can do it for You.
Customization means that you have the logo in which you want to change such as font, color, or non conceptual change some elements, convert to black/white version.
Maybe you need exclusive logo?
Also in my portfolio you can find gorgeous fantasy backgrounds, web elements, textures, beautiful flyers for different events, etc.
Layered: Yes; Graphics Files Included: Transparent PNG, Vector EPS, AI Illustrator, CorelDRAW CDR; Resolution: Resizable; Minimum Adobe CS Version: CS.
Keywords: arrow, arrows, art, business, community, connect, cosmos, design, direction, evolution, fast, geek, general, globe, idea, mind, people, progress, round, social, space, speak, talk, tape, tech, technology, union, up, web, world.
Create GraphicRiver Ideas Generator Logo 5687571 style with After Effect, Cinema 4D, 3DS Max, Apple Motion or PhotoShop. GraphicRiver Ideas Generator Logo 5687571 from creative professional designers. Full details of GraphicRiver Ideas Generator Logo 5687571 for digital design and education. GraphicRiver Ideas Generator Logo 5687571 desigen style information or anything related.


text:
Are Artificial Sweeteners good for weight loss?
Excessive sugar consumption is now recognized as being a major contributor to increasing rates of obesity (1), metabolic syndrome (2), diabetes(3), high cholesterol, heart disease (4, 5, 6), and even cancer (7, 8, 9,10).
In response, many people turn to artificial sweeteners for use in soft drinks, candies, baked goods, ice cream and other sweet foods.
Since artificial sweeteners are near to zero calorie, you might think they would clearly aid weight loss, but that is debatable (11)
Besides weight loss, many Doctors and Nutritionists believe artificial sweeteners are in fact very bad for your health (12,13).
We do not cover the natural low-calorie sweeteners such as stevia, xylitol, erythritol, sorbitol and mannitol in this article.
Artificial Sweeteners and Appetite Regulation
Sweet foods activate pathways in our brains that cause the release of hormones. This is known as the "food reward" pathways (14, 15, 16).
The satisfied feeling we get after eating sweets is due to this release of hormones, which is similar to other addictions like drugs (17, 18, 19).
Artificial sweeteners fool our taste buds, but do not activate the same pathways in our brains to trigger the food reward. As a result, cravings for more sugary foods is increased (20).
The hypothalamus is the appetite regulator in the brain. Magnetic imaging shows decreased activity in the hypothalamus after consuming sugar drinks (21).
This decreased activity in the hypothalamus is not seen after consuming aspartame, indicating the food reward pathways have not been as fully activated.
The sweet taste is rewarding, but without the satiety it may be that you continue craving and end up consuming more calories.
Different Types of Artificial Sweeteners
1. Aspartame – (NutraSweet, Equal)
The FDA approved aspartame over 35 years ago. Today it's found in over 6,000 foods and drinks. It isn't heat-stable, so is not used in baked foods.
2. Sucralose (Splenda)
Sucralose is 600 times sweeter in sugar. It is used in tabletop sweeteners and a variety of beverages, dairy products, fruits, syrups, and baked goods.
3. Acesulfame K (ACE, Sunette, Sweet 'N Safe)
Acesulfame K is used in sugar-free chewing gum, candy, dairy products, and alcoholic beverages. It's also increasingly popular as a "flavor enhancer."
4. Saccharin (Sweet 'N Low)
In the 1970s, saccharin was thought to cause bladder cancer. This has since been disproven and the FDA has rescinded warning about it. Currently it's used to sweeten children's medications such as chewable aspirin, cough syrup, and other medications.
Increased Sugar Cravings
Opponents of artificial sweeteners sometimes argue that they increase sugar cravings (22).
The reasoning is that the option of a zero calorie diet soda causes people to drink more sodas than if the only option is a sugar filled 200 calorie soda.
There are no calories, and they satisfied their craving, so isn't that a good thing?
Well, we know that our tastes are trained so we prefer more of things we eat regularly (23).
The more sweets we eat or drink, the more sweets we crave.
The reverse is also proven to be true. For example, reducing salty or fat foods for a few weeks lowers our cravings for salty or fat foods (24, 25).
Artificial Sweeteners and Metabolic Health
Some studies suggest frequent use of artificial sweeteners actually increase the risk of metabolic syndrome and type 2 diabetes (26)
One study found that they increased risk of metabolic syndrome by 34% (27).
Another study found that diet sodas increased the risk of type 2 diabetes by 121% (28).
How these increased health problems occur may be explained by a recent study that shows zero calorie diet sodas alter the bacteria in our stomach (gut microbes) and induce glucose intolerance (29).
These gut microbes are key as there is evidence that many chronic metabolic diseases begin in gut, and that probiotics supplements can help weight loss (30, 31).
Artificial Sweeteners aid Weight Loss
In studies where caloric intake was closely monitored and controlled, it is clear that use of artificial sweeteners instead of sugar does aid weight loss (32, 33).
Reviews of dozens of clinical trials found use of artificial sweeteners instead of sugar resulted in weight loss of about just under 2 pounds on average (34, 35, 36).
The largest study tracked 641 children of 4-11 years for 18 months.
Every day at lunch they were given 8 ounces of a high calorie sugary drink or a zero calorie artificially sweetened drink.
Those assigned to the zero calorie drink gained on average 2.2 pounds less than the group that consumed the sugar drink (37).
Final Thoughts
Artificial Sweeteners might not be as effective in satisfying cravings for sweets, and may even increase the cravings.
They may also contribute to health problems such as leaky gut syndrome, although no definitive proof yet exists.
However, if portions are controlled and they are simply used to replace a sugar filled soda or fruit juice, and do not encourage greater consumption, then artificial sweeteners have a slight positive effect in aiding weight loss.
The weight loss benefits are not overwhelming so the choice of sugar or artificial sweeteners is not critical for weight loss success.
The research IS overwhelming that excessive sugar consumption leads to myriad health problems, so your best bet is to avoid sweet foods and drinks as much as possible.
- After being laughed or scoffed at for decades, ever since Dr Atkins popularized it, there are dozens, if not hundreds of different takes on the low carb diet now.
- We've been huge fans of Berberine for multiple health benefits. It's now being included in weight loss products are well.
- Probiotics are much like Berberine – just now becoming known to the mainstream, and also showing promise for weight loss.
- Cumin is very well known as a spice that studies are showing can aid dieters drop a little extra weight.
- Niagen is being called the "God Vitamin" by some for its anti-aging and health regeneration properties.
- The best known diet pill of the last few years is still the king, but so many scammers selling worthless Garcinia Cambogia on Amazon have definitely hurt its reputation.


text: No, you are misinterpreting. And I am not so sure why this matters. As we are all trying to help this person out, but ok, I will explain so you will get off my case. I have used the sit method before, if fact, it was what we first tried. We have found than when a person enters, he gets more gratification from jumping than he does from a treat. So YES, when we use that particular method it does not work and he continues to jump. THAT was what I was saying. I turned around and started using the ignore method because I have heard and seen that work before, from you know who. And that has worked much better for US. Especially when a NEW person enters is when he likes to jump, but it very quickly ceases when he gets the opposite of what he wants... ATTENTION! Now when familiar people come over, he rarely ever tries to jump, because he knows where it will get him. With new people, he needs a reminder, that they WILL ignore him as well. Nope, Reggin is not big. EDIT: It is a work in progress.


text:
Services for entrepreneurs
provided by public and private institutions from Catalonia
Credits for social enterprisesAssociació FIARE Catalunya
Funding | Loans and guarantees
FIARE finances social, environmental, cultural, humanitarian and international cooperation projects. To be awarded credit, individuals or institutions that apply for financing must fulfil two analysis criteria: one is economic whereby its purpose, its social impact and its capacity to repay the credit is evaluated; the other involves determination of whether the social and environmental impact fulfils the principles of the credit cooperative.
Addressed to: Third sector and social economy institutions and social enterprises that wish to establish a company.
This service may be of use to you if ...
you are in the phase of:
Creation , Growth and consolidation
Your business is:
Social enterprises and cooperatives
You are in the sector:
All sectors.
Regional service, access
Carrer de Sant Isidre, 140
08208 Sabadell
08208 Sabadell
933 689 982
Consult website
933 689 982
Consult website
Carrer Casp, 43
08010 Barcelona
933 689 982
08010 Barcelona
933 689 982


text:
Mick's Blog
Hey guys, have been on a mission lately! Feels like I've been around the world 20 times and all in three weeks. Love it!! First up was the Brazil WT, waves were pretty fun and ended up with 9th place; lost a close heat to CJ. Had fun though, all the boyz went mad for a night, was sick!
Then packed bags to SA... J-Bay was flat at first but then turned on for the last three days. Was feeling good and surfed well but was beaten by SA super wildcard Sean Holmes! I fell in a couple of crucial tubes on the end section, which cost me, plus Sean surfed real well. I got a 9th there. Just got back from Bali yesterday. How good is that joint? Went to Scar Reef to catch a swell, plane to Lombok, car ride to ferry, ferry ride to car, then to camp. Fun 4-6ft waves, surfed for three days solid.
Off again
Then back home for a few days in Port Macquarie before heading off to Costa Ricafor World ISA Games. Keep you cats posted! Cambo


text:
Irelia Build Guide by iCondomMan
Not Updated For Current Season
This guide has not yet been updated for the current season. Please keep this in mind while reading. You can see the most recently updated guides on the browse guides page.
Thank You!
Your votes and comments encourage our guide authors to continue
creating helpful guides for the League of Legends community.
Irelia Build
Ability Sequence
Not Updated For Current Season
The masteries shown here are not yet updated for the current season, the guide author needs to set up the new masteries. As such, they will be different than the masteries you see in-game.
This is a guide about how to play Irelia. This build is very useful because it gives you tons of damage and durability. With this build I made a penta very easily. It worth a try ;)
Start with a doran's shield. Then buy mercury treads, trinity force, warmogs, atmas, bloodthirster and end with a black cleaver.
Summoner Spells
For summoner spells take Exhaust and Flash so you will be able to kill enemies easily and escape from difficult situations.


text:
Remington UMC Folding Hunter USA NOS R3 big Game
Sold for:
Start Free Trial
or Sign In to see what it's worth.
- Sold Date:
- Source: eBay
Remington R3 Lockback w/Locking Bonesaw. 5" closed.High carbon steel Clip blade and bone saw. Brown jigged Delrin handleswith nickel silver bolster and inlay shield. Lanyard hole.Knife also has Brass Liners and Pins.This knife is mint NOS. New/Old stock with the Tang stamp Remington UMC Made in USA.andR-3 Patented.Ithink this knife was made in the 60's -70's but could be older due tothe UMC tang stamp.Was made by Camillus N.Y back in the day forRemington Company.
Posted with eBay Mobile


text:
Tampa Pest Control
We take great pride in offering our stellar customer service at every level in the tampa pest control service industry.
We have a reputation for being one of the most progressive pest control companies in the industry by putting the latest and most innovative pest control techniques to work for you in Pest Control. Be prepared to be wowed by our courteous staff and excellent service. This commitment is what sets us apart from our competition.
Prohealth Pest Control serves Tampa and surrounding areas including Brandon, Clearwater, St. Petersburg, Safety Harbor, Palm Harbor, Ybor City, Plant City, Valrico etc.


text:
My Nonprofit Reviews
231 profile views
1 reviews
Review for Farm Fresh Rhode Island, Pawtucket, RI, USA
Farm Fresh RI makes me proud to live in RI. Not many states are able to empower their farmers and create a network that works so smoothly to connect farmers to markets, restaurants and other outlets. Keep up the good work!
More Feedback
I've personally experienced the results of this organization in...
Providence, RI
If I had to make changes to this organization, I would...
Continue the growth.
Role: General Member of the Public & I buy my produce at the Winter Farmers Market.


text:
Honours were almost completely even at stumps on day two of Glamorgan's LV= County Championship Division Two clash with Gloucestershire at Cardiff.
Jim Allenby scored 85 for the hosts as they reached 271-9 in response to Gloucestershire's first-innings effort of 275 all out.
Gloucestershire had resumed their first innings on 228-8 and added a further 47 runs in almost 23 overs before Allenby (3-31) finally mopped up the tail.
Glamorgan then suffered a setback in their reply as they found themselves reduced to 47-3, Gareth Rees (7) edging Graeme McCarter into the slips, while Ben Wright (0) and Murray Goodwin (14) also failed to get going.
But Chris Cooke and Allenby rescued the Welsh county from their shaky predicament, putting on 56 for the fourth wicket despite the latter being dropped early in his knock.
Cooke also played well in completing a 64-ball half century but, without adding any more runs, Matt Taylor (3-98) broke through his defences to leave Glamorgan on 103-4.
After tea, David Lloyd perished for 16 when he edged to Chris Dent at second slip off Will Gidman (4-64), who then had skipper Mark Wallace (7) caught behind.
But Allenby continued, completing a 68-ball fifty but falling 15 runs short of a century when he edged Taylor to Dent to give the slip fielder his fourth catch of the innings.
Ruaidhri Smith (39) was the ninth man out before Dean Cosker (19 not out) and Michael Hogan (16no) put on an unbeaten 27 for the final wicket by stumps.


text:
My husband and I went to Inase for a week-night meal on Tuesday. We had a really lovely meal and some of the best sushi we've EVER eaten. The cooked appetisers were delicious as well. The service was muted, courteous, attentive and considerate (all good things). The atmosphere was very soothing. We sat at the sushi bar and the sushi-chef was very nice.
An important disclaimer given I'm posting on CH - we aren't sushi experts like some posters here. We've never sampled Yasuda or East 15.This was only our second time trying omakase (the first being the omakase sushi meal at Takesushi in Midtown. But we both love sushi and will try it where we can get it.
We both ordered the $60 omakase meal.
We started with standard appetisers, miso soup, some kind of seaweed, salad in ginger-orange dressing. Each perfectly formed. If someone can identify the seaweedy thing, please let me know what it is called.
The waitress recommended the chawan-mushi (egg custard) with uni (sea urchin roe) as part of our meal. It contained perfectly done little pieces of shrimp and gingko nut. The chicken pieces were slightly overcooked. "So many different flavours," whispered Hubby. (The restaurant was REALLY quiet.) "Yeah, how'd they get it all into such a little cup?" I whispered back. The uni tasted like a clean, briny ocean breeze.
After the appetisers, we each got this beautiful platter of sashimi. The chef took pains to tell us what everything was, and that the red snapper was from Japan. There was Spanish mackeral, medium-fatty tuna, red snapper, octopus, abalone, crab, abalone and fresh-grated wasabi. Everything was really fresh, tender and tasty. The tuna was my favourite. It was extraordinarily flavourful.
This was followed by a yummy yellow-tail and turnip stew. The fish was almost buttery.
Sushi followed. We were served two pieces at a time, totaling eight pieces for each of us. (There are only 6 pieces in the photos because I got excited and forgot to take a photo of the first two.) Sushi included fatty tuna, fatty salmon, scallops, raw(!) giant shrimp, horse mackeral and salmon roe. We liked everything and thought each piece was perfectly seasoned. Biggest surprise: the giant shrimp, which was raw with plenty of wasabi. I didn't think I would like raw shrimp. But this was unctuous rather than slimy, with a subtle prawn flavour. My favourite: the tuna, the tuna, the tuna! Who knew tuna could be so tasty. The fragrant, oily fish taste just kept on going in my mouth.
We finished with matcha and red-bean mochi ice-cream, both good.
We thought the food and value was better than Takesushi, although I think they are different kinds of restaurant. Inase had fewer "interesting" options like soft shell crab and lobster tempura. A smaller, warmer and somehow more homey menu and experience, although that does *not* mean the food was less expert.
1586 1st Ave, New York, NY 10028
1026 2nd Ave, New York, NY 10022


text: PESHAWAR: The Khyber Pakhtunkhwa Senior Minister for Social Welfare and Women Development Sikandar Hayat Khan Sherpao has said that the present government is taking serious steps for improvement of the state of human rights in the province by provision of legal coverage.The Minister called upon the International Community to come-forward and support the efforts of the Provincial Government for empowerment of women, children, special persons and senior citizens.He was addressing the inaugural ceremony of toll free "Bolo Help Line 0800-22227" established for supporting women rights at the Directorate of Social Welfare in Peshawar. The occasion was graced by the Australian High Commissioner to Pakistan, Margaret of ADAMSON, John O' Brein Country Director Trocaire, Representative of Aiming Change for Tomorrow (ACT), Officers of Social Welfare Department and a large number of representatives from National and International NGOs working for the women right.The Provincial Minister termed the establishment of the said Help Line as a fundamental step for provision of rights to the women, supporting the victims of human rights abuse at every level against criminals. The Minister added that the helpline would help women-folk in integration as the vital part of society.The Provincial Minister while highlighting the efforts made for the betterment of all the down trodden segment of the society stated that legal cover was being provided as per the international requirements, aiming to bring them at par with others.He said that a survey was being conducted to get the detailed database of victims, special children and senior citizens of the whole province aiming to facilitate them. He also directed the concerned quarters to keep the information of victims confidential and provide all possible help to them.The occasion was also addressed by the Australian High Commissioner, Country Director Trocaire, ACT Chief and others who highlighted the importance of the said help line and appreciated the efforts of the Provincial Government. Earlier, Fateha was offered for the victims of the plane crash and the gathering expressed sympathies with bereaved families.


text:
We recommend upgrading to the latest Internet Explorer, Google Chrome, or Firefox.
If you are using IE 9 or later, make sure you turn off "Compatibility View".
Handpicked brands for you
So cool to see our girl @skingshill take the win in the pro class at Fontucky alongside this random! |


text:
VideoHive Small Fire Bar 6827973
Stock Footage \ Nature
Flames \ Effects \ Visual \ Clipping \ Entire \ Lighten \ Screen \ Compositions
Stock Footage - Small Fire Bar 6827973 by Fermu @ VideoHive
Stock Footage \ Nature
2 Full HD real small fire bar and flames shots. No sound. Flames stay in the frame the entire time so there is no clipping at the edges. Black background easy for 2D, 3D and visual effects compositions (lighten,screen)
Last Update: ; Alpha Channel: No; Looped Video: No; Frame Rate: 29.97; Resolution: 1920x1080; Video Encoding: H.264; File Size: 205mb; Total Clip(s) Length: 0:46.
Real Fire Torch with Large Flames Prekeyed; Real Fire Torch Transition Prekeyed Pack 3; Real Fire 02; Real Fire Looped Prekeyed; Brain in a Water Tank with Oxygen 03; Brain in a Water Tank with Oxygen 02; Brain in a Water Tank with Oxygen.
Keywords: 3d, bonfire, burn, composition, fire, flames, fuego, fx, gas, hoguera, orange, physical, practical effects, pyrotechnic, visual effects.
Download Link


text:
Here goes...who wants to talk about your next
View Single Post
June 26th, 2006, 09:57 AM
Platinum Supermommy
Join Date: Oct 2005
Location: Ontario, Canada
Posts: 18,680
I'm scared of...
- Having another ectopic
- Having another m/c
- Having to deal with family & friends who think they can relate when none of them have ever dealt with a lose and have no clue what I'm going through.
- Finding out there's no heartbeat after hearing one
- Finding out something is wrong after the baby is born
- I'm terrified that I'm not meant to be a mom and these 2 losses were God's attempt to tell me this and I wouldn't listen.
- I'm scared to tell people that I'm pg when it happen again
- Of running into people that I know who have healthy babies and didn't want them in the first place
- Of dealing with a friend who's due in September and hearing here play by play of everything even though she knows what happened to me.
- Of having irregular cycles once AF shows up
- Of finding out that the Endo is worse and my hormones are out of whack again for the other medical condition that I had
- Of finding out that there's something wrong with either DH or myself and that's causing the losses.
I'm just plain scared!!!
Thank you Claire1977 for my adorable siggy
...And All That's Nice
For the special little one in your life!
View Public Profile
Find all posts by Nykoal


text:
I suppose October 31st is close enough to November to start releasing the films that will be gunning for awards. Today was the first time in 2014 that I saw a movie and thought 'Wow, I can see this getting Academy Award nominations.'
That movie? Nightcrawler.
No, it's not a Marvel movie about Kurt Wagner. It's much, much weirder than that.
Starring Jake Gyllenhaal as Louis Bloom, the film follows his attempt to become a successful freelance cameraman. Fortunately for him he's a hard worker and a fast learner.
As a man without a job, Louis resorts to stealing fencing and copper wiring so that he can resell them for enough to pay the bills. A two-bit conman, his saving grace is that he can talk and he can listen and knows when to do each. A chance encounter with a freelance film crew (led by Bill Paxton) at the scene of an accident gives him an idea for a new job. No boss and a potential paycheck just for getting gruesome scenes on tape? Sounds great!
One stolen item sold to a pawn shop later and he's on the road with a handheld camera and a police scanner. The only problem is that the job is not as easy as it seemed. Sure, filming a crime scene merely requires being at there, but getting there before the police shut down the area is tough. After several failed attempts he finally comes up with the type of footage that will actually get him paid. But Louis knows he'll need help to really succeed. Enter Rick (Riz Ahmed), a young man desperate for cash. He accepts the offered $30/day salary and becomes the navigator and assistant.
This starts working out very well, to the point that Louis begins selling footage regularly to Channel 6. In doing so he develops a working relationship with Nina (Rene Russo), the woman in charge of deciding what is and isn't news. But that's still not enough. The rest of the film gets more and more seedy and creepy as Louis does literally (yes, I'm using it correctly here) anything he can to score better footage to make more money.
What makes Nightcrawler work is Jake Gyllenhaal. He plays a creepy sociopath with no regards for other people and is frighteningly good at it. He mentally abuses Rick, forces Nina to give in to his every demand, and does everything in his power to beat the competition. On top of all that he's not afraid to manipulate the footage to maximize his earnings.
While there are a lot of people 'in' the movie, it almost exclusively focuses on Gyllenhaal, Rene Russo and Riz Ahmed. I'll give it to two out of the three, they're complex, interesting characters. It's just that Jake Gyllenhall obliterates them and dominates nearly every single shot. My hat goes off to him for giving one of the creepiest, most compelling performances I've seen in recent years.
Between the acting and a genuinely disturbing script, Nightcrawler is one of the best dramas I've seen in a long time and currently my favorite film of the year.
Top image © Open Road Films.


text:
Kukla's Korner Hockey
Rumor Update
by Paul on 12/07/05 at 07:53 PM ET
Comments (0)
Filed in: | KK Hockey | Permalink
Be the first to comment.
Add a Comment
Please limit embedded image or media size to 575 pixels wide.
Add your own avatar by joining Kukla's Korner, or logging in and uploading one in your member control panel.
Captchas bug you? Join KK or log in and you won't have to bother.
Most Recent Blog Posts
Going As Planned For The St. Louis Blues
Video- You Make The Call Coyotes/Leafs
Video- Wayne Gretzky Was Close To Signing With The Toronto Maple Leafs
Marc Methot Out Another 4-6 Weeks
About Kukla's Korner Hockey
Paul Kukla founded Kukla's Korner in 2005 and the site has since become the must-read site on the 'net for all the latest happenings around the NHL.
From breaking news to in-depth stories around the league, KK Hockey is updated with fresh stories all day long and will bring you the latest news as quickly as possible.
Email Paul anytime at email@example.com


text:
SITS:Vision - Don't just take our word for it
Arts University BournemouthThe 'push and pull of data' at Arts University Bournemouth
Arts University Bournemouth (AUB) is a leading specialist UK university offering high quality specialist education in art, design, media and performance across the creative industries.
AUB recently purchased a third-party software platform and needed it to integrate with their student information sy…
University of the Arts LondonUniversity of the Arts London set to transform the student journey
In October 2016, the University of the Arts London (UAL) awarded Tribal Group plc a contract for the design, installation and implementation of the higher education student management system, SITS:Vision.
Earlier in the year, UAL invited bids for a replacement student system as their existing st…Read the case study
Regent's University LondonTribal has been working in close partnership with Regent's University London (RUL) to make the finance process easier for both staff and students. Using proven software and expert in-house knowledge it is now much simpler for students to manage their fees and for staff to process them.
Regent's University London is London's only independent, not-for-profit university. 3,500 students from 140 different countries form a highly cosmopolitan community. Based in the heart of Regent's Park, the University has a rich and diverse history, and a long tradition of education and achievement.…Read the case study
University of YorkTribal's SITS:Vision was chosen by the University of York to improve their student administration process.
A member of the elite Russell Group of universities, the University of York has established a reputation for excellence in teaching and research, such that it is now recognised as one of the UK's leading universities. Opening in 1963 with just 230 students; there are now over 30 academic departments…Read the case study
University of Wolverhampton
The University of Wolverhampton's technical support contract with Tribal provides valuable insurance for the maintenance and support of their systems.
In an environment where critical data is in high demand, there is increasing pressure within HE for institutions to manage and maintain complex tech…Read the case study
University of LondonWhen one of the UK's oldest universities wanted a student admissions system fit for a 21st century world, they turned to Tribal...
The University of London was founded by Royal Charter on 28 November 1836, and is the third oldest university in England. Today, more than 22,000 students study at its 18 colleges across the capital. Unsurprisingly, managing student admissions each year is a huge logistical challenge, and any proble…Read the case study
Cardiff UniversityA New Way Forward for Programme Development - Interview with Rhodri Evans 5 November 2014
About Cardiff University
Established in 1883, Cardiff University is now the 12th largest university in the UK in terms of student numbers with over 28,000 people studying more than 300 degree programmes. The university is the only Welsh member of the Russell Group and is aiming to be one of the wor…Read the case study
University of Oxford
The system needed to provide effective support for teaching and learning activities, including the registration of students and administration of their academic progress.
The requirement
The existing systems data and functions were supplemented by numerous other systems across the University that …Read the case study


text:
Category: Stock Graphic, Product Mockups
Stock Graphic >> Product Mockup - RUG MOCK-UP 226794 by nightmarecatcher @ CreativeMarket
File Size: 39.28 MB; Layered: Yes.
- 2 psd files 2000x2000 px 300 dpi;
- 1 View;
- You can put your own design using smart objects;
- You can coose gradient and color;
- Realistic effect with displace maps;
- well oganized layers.
Keywords: mock up, mockup, mock-up, template, bed, bedclothes, curtain, pillow, bedroom, rug, roller, blind, pillows, curtains, home.