1. A method comprising: setting parameter values for a set of acoustic models used in speech recognition; for each of a set of utterances, a decoder in a computing device decoding the utterance using the set of acoustic models to identify a set of competitor word sequences for the utterance and to determine a probability of each word sequence given the utterance; setting a weight with a positive value that is different than one for at least one competitor word sequence of each utterance; updating a parameter value in the set of acoustic models using a trainer through steps comprising: for each competitor word sequence for a selected utterance, using the weight for the word sequence, the probability of the word sequence given the selected utterance and an occupation probability that describes the probability of being in a particular state of the acoustic model at a particular time given the selected utterance and the word sequence to form a score for the word sequence; summing the scores for the word sequences as part of forming a score for the selected utterance, wherein forming the score for the selected utterance further comprises determining a term ∏ r = 1 R ⁢ ⁢ ∑ s r ⁢ ⁢ w ⁡ ( s r ) ⁢ p Λ ′ ⁡ ( s r ❘ X r ) ∑ s r ⁢ ⁢ w ⁡ ( s r ) ⁢ p Λ ′ ⁡ ( s r ❘ X r ) , where p Λ′ (s r |X r ) is the probability of word sequence candidate s r given selected utterance X r using parameter values Λ′, w(s r ) is a non-negative weighting factor for competitor word sequence s r , and R is the total number of utterances in the set of utterances such that the term is equal to the product of weighted sums for all of the utterances except the selected utterance X r ; and using the score for the selected utterance to update a parameter value.