1. A device comprising: a profile building component in communication with an electronic data store; a speech recognition component; and a sensor configured to detect movement of a user independent of a direction of the user's gaze and without detecting physical contact between the user and the device; wherein the profile building component is configured to: receive, from the sensor, an indication that presence of the user was detected; begin listening for utterances from the user in response to receiving the indication; detect a first voice signal corresponding to a first utterance of the user; determine an identity of the user using the first voice signal; process the first voice signal to determine acoustic information about the user, wherein the acoustic information comprises at least one of an age, a gender, an accent type, a native language, or a type of speech pattern of the user; perform speech recognition on the first voice signal to obtain a transcript; process the transcript to determine language information relating to the user, wherein the language information comprises at least one of a name, hobbies, habits, or preferences of the user; store, in a user profile associated with the identity of the user, the acoustic information and the language information; determine acoustic model information using at least one of the first voice signal, the acoustic information, or the language information; and determine language model information using at least one of the transcript, the acoustic information, or the language information; and wherein the speech recognition component is configured to: receive a second voice signal corresponding to a second utterance of the user; determine the identity of the user using the second voice signal; perform speech recognition on the second voice signal using at least one of the acoustic model information or the language model information to obtain a word sequence that indicates that a third utterance corresponding to a language characteristic will be uttered by a second user different than the user at a time after a current time; and select a second user acoustic model corresponding to the language characteristic for performing speech recognition at the time after the current time.