1. A computer-implemented method of developing a pattern recognition model, the method comprising: training, using a processor, a plurality of models for the pattern recognition model with diagonal covariance matrices, each model comprising a continuous density hidden Markov Model; building, using a processor, a covariance hierarchical tree structure of states from the plurality of models wherein the covariance hierarchical tree structure includes a root node and children nodes associated with the root node, and a plurality of leaf nodes, each leaf node representing a Gaussian component of the continuous density hidden Markov Models; generating, using a processor, a diagonal covariance matrix for the root node and children nodes prior to a leaf node, without estimating a full covariance matrix; creating, using a processor, a full covariance matrix for each leaf node in the hierarchical tree structure for each of the plurality of models by estimating terms of the full covariance matrix based on related models using the hierarchical tree structure and wherein creating the full covariance matrix for each leaf node includes linearly combining the generated diagonal covariance matrices for the nodes along an upward path from a given leaf node to the root node, using a set of combination weights, to estimate off diagonal components of the full covariance matrix for the given leaf node; replacing, using a processor, the diagonal covariance matrices of the leaf nodes in the plurality of models with the full covariance matrices to modify the pattern recognition model; and storing, using a processor, the pattern recognition model on a non-transitory storage medium for use in a pattern recognition system.