Using HMM for speech recognition
What exactly can the vector/observation be? Can it be a relatively simple function, say approximation of the spectral envelope? Or is it usually a vector of R^n?


You won't go very far by summarizing the entire spectral envelope of a sound by a single real, so yes, observations are vectors - their dimensionality is in the 10 to 100 components bucket...

Typically, the features used are MFCC and their first/second order derivatives. They capture spectral envelope (in short: dimensionality reduction applied to a low-resolution spectrum computed on a perceptual scale) and have several invariance properties that make them relatively robust.


  How is it, that if you say "ooone" instead of "one", P("ooone"|M_1) will still be the highest of the three trained HMM models? Is it because if the model was trained with 'one', the model already favors the observation that happens when the sound of 'o' is said, and the state transition of x_1 -> x_1 (where x_1 is likely to emit this observation) is also favored?


Yes, this is what happens.

The cost of stretching the "o" is high, but still lower than having the emission probabilities totally wrong by matching the "o"s or the "n" that follows with other phonemes.


  Basically, is there some convincing proof/intuition, that pronouncing a certain word very slowly at certain parts will not cause it to be recognized incorrectly?


If the features used for the observation vector are robust, and if the model is trained on the same speaker as for recognition (or if suitable model adaptation measures are used), the emission probabilities can get very low when confusing phonemes.

One thing that could cause confusions would be a HMM trained on utterances of "five" and another trained on utterances of "fiiiiiiiiiiile". In this case, "fiiiiiiiiiiiive" might be recognized as "fiiiiiiiiiiiile" - the cost of repeatedly stretching the "i" might be enough to counterbalance the confusion from "v" and "l" on a handful of frames. So an abnormal utterance could be confused with another abnormal utterance for which a model is available. The thing is, abnormal utterances are "averaged out" during the training process.


  does the classification HMM operation happen at all?


For recognition, it is not relevant to know the most likely sequence of states. However, there are applications for which knowing the sequence of state is needed.

In particular, we sometimes want to exactly synchronize the recognized sequence with the original audio recording. This could be useful for subtitling, or recording a timestamped transcription of a recording. This is also used during Viterbi training, an alternative to the Baum-Welch algorithm that uses state decoding (classification) in the training loop.


  Lastly, could some please explain how continuous speech recognition differs from this?


Left-right models with a handful of states are used to describes diphones or triphones. States have hundreds to thousands mixture components (or nowadays we eschew GMMs and use other emission models). This is already a ridiculously large model space, so there are tied-states - that is to say some states in the model share the same emission probabilities with other states in the model. Anything that pools together the parameters from several parts of the model ensures that more training data will be used.

Then, the triphone models are concatenated together to build word models - using pronunciation dictionaries (lexicons). The lexicon maps a word into a sequence of phones. While there are approaches to automatically learn it, this data primarily comes from linguists.

Then word models are concatenated together to build a language model. The result is an extremely big FST, which is not even fully composed in memory from its component - but only traversed on the fly during recognition.

Of course the whole thing is not trained with Baum-Welch... We train separately each level... Tri-phone HMMs are trained on aligned speech data; lexicons are curated by linguists; language models trained separately on purely textual data...

Typically a small amount of aligned speech data (in which the transcription are aligned with the audio) is used to bootstrap the triphones model. Then this inexact model can be used to align a larger set of unaligned training data, and the process is iterated.