1. A method, comprising: receiving input speech to be decoded; converting a non-back-off language model that assigns a probability to each fixed order n-gram generated with a given vocabulary to a back-off language model that assigns a probability to one or more fixed order n-grams assigned a zero probability by the non-back-off language model by backing off to a given lower order n-gram corresponding to the given fixed order n-gram; and pruning the converted back-off language model; and utilizing the converted back-off language model in a speech decoding process to generate a decoded speech output from the input speech; wherein the converting step comprises, starting with an initial non-back-off language model associated with a lowest order n-gram and hierarchically progressing with one or more higher order non-back-off language models until the fixed order n-gram is reached, converting the initial non-back-off language model and the one or more higher order non-back-off language models into respective back-off language models; wherein the given fixed order n-gram comprises a given word and an associated history; wherein the given lower order n-gram corresponding to the given fixed order n-gram comprises the given word and a truncated version of the associated history; and wherein the method is performed utilizing at least one processing device comprising a processor coupled to a memory.