1. A method comprising: receiving an input having at least one speech segment and at least one non-speech segment; establishing a first restriction of recognizing only speech states during the at least one speech segment; establishing a second restriction of recognizing only non-speech states during the at least one non-speech segment; generating, via a processor, a hypothesis lattice, wherein the hypothesis lattice allows any sequence of speech states and non-speech states; and generating a reference lattice, wherein the reference lattice is based at least in part on the hypothesis lattice and conforms to the first restriction and the second restriction.