1. A computer-implemented method for creating and using a compact finite state transducer (FST) for automatic speech recognition (ASR), the method comprising, during a training period: configuring a representation of the FST, wherein the representation comprises: a first table representing at least a portion of states of the FST, the first table including a first record representing a first state of the FST, the first record comprising: a representation of a weight of the first state, and a first reference to a second record in a second table, the second record representing at least one first arc, the first arc representing a connection from the first state to another state in the FST; the second table representing at least a portion of the arcs of the FST, the second table including the second record representing the first arc, the second record comprising: a representation of a weight of the first arc, a bit indicating that the second record lacks an indication of a next state, a first representation of the input label of the first arc, and a second representation of the output label of the first arc; and performing ASR using the representation of the FST, wherein performing ASR comprises: receiving audio data comprising audio feature vectors for ASR; loading the representation of the FST into a memory; performing ASR on a first portion of the audio data using the first record and the second record; determining that the bit indicates that the second record lacks the indication of the next state; determining, using information about the first state, an estimated destination of the first arc; determining that the estimated destination is a second state of the FST; and performing ASR processing on a second portion of the audio data using a third record in the second table, the third record representing the second state.