1. A computer-implemented method comprising: receiving audio data encoding an utterance, and data specifying a time when the utterance was spoken; determining, for each of multiple communications that were initiated by a user of a mobile device, a time when the communication was initiated or received; determining, for each of the multiple communications, a similarity score based on a similarity between the time when the communication was initiated or received, and the time when the utterance was spoken; determining, for each of multiple contacts associated with the user, a probability associated with the contact based at least on the similarity score for the communications that were initiated or received; weighting a contact disambiguation grammar according to the probabilities; and processing the audio data using the contact disambiguation grammar to select a particular contact.