1. A method of operating a pattern recognition system for refining a plurality of statistical pattern recognition models that are used for statistical pattern recognition, the method including: reading in initial values of a set of parameters for said plurality of statistical pattern recognition models; reading a training data set that includes a plurality of training data items including training data items for each of said plurality of statistical pattern recognition models, along with a transcribed identity for each of said plurality of training data items; obtaining feature vectors from each of the plurality of training data items; using a processor to perform an optimization routine for optimizing an objective function in order to find refined values of said set of parameters for said plurality of said statistical pattern recognition models corresponding to an extremum of said objective function, wherein said objective function is dynamically defined for each of a succession of iterations of said optimization routine to include a subexpression for each k th item of training data in, at least, a subset of said plurality of training data items that is defined by, at least, a first criterion that requires that said transcribed identity does not match a recognized identity for said k th item of training data, and a second criterion that requires that there is not a gross discrepancy between said transcribed identity and said recognized identity, wherein each subexpression depends on a relative magnitude of a first probability score compared to a second probability score, wherein said first probability score is based on a value of a first statistical pattern recognition model corresponding to said recognized identity of said k th item of training data evaluated with said one or more feature vectors obtained from said k th item of training data and said second probability score is based on a value of a second statistical pattern recognition model corresponding to said transcribed identity of said k th item of training data evaluated with said one or more feature vectors obtained from said k th item of training data; and using the refined statistical pattern recognition models to recognize a pattern.