15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising: receiving an input having a speech segment and a non-speech segment; establishing a first restriction of recognizing only speech states during the speech segment; establishing a second restriction of recognizing only non-speech states during the non-speech segment; generating a hypothesis lattice, wherein the hypothesis lattice allows any sequence of speech states and non-speech states; and generating a reference lattice, wherein the reference lattice is based on the hypothesis lattice and conforms to the first restriction and the second restriction.
17. The non transitory computer-readable storage device of claim 15 , the computer readable storage device having additional instructions stored which result in the operations further comprising employing the reference lattice as a barge-in model for speech processing.