1. A method of authenticating a telephone caller, the method comprising: receiving, by a processor of an authentication server, audio data including speech of the telephone caller; analyzing, by the processor, the audio data to identify a plurality of words from the speech of the telephone caller and to identify an occurrence frequency for each of the plurality of words; comparing, by the processor, the plurality of words and the occurrence frequencies to a plurality of word clusters, each word cluster comprising a plurality of associated words and an occurrence frequency for each of the plurality of associated words, and each word cluster being associated with one of a plurality of demographics; determining, by the processor, a most similar word cluster of the plurality of word clusters to the audio data based on a similarity of the plurality of words and the plurality of associated words of the most similar cluster and a similarity of the occurrence frequencies of the plurality of words and the occurrence frequencies of the plurality of associated words of the most similar cluster; receiving, by the processor, a purported identity of the telephone caller, the purported identity including caller demographic data; comparing, by the processor, the caller demographic data to the demographic associated with the most similar word cluster; and identifying, by the processor, the telephone caller as at least one of: likely having the purported identity in response to determining the caller demographic data matches the demographic associated with the most similar word cluster, and unlikely to have the purported identity in response to determining the caller demographic data matches a demographic associated with a word cluster different from the most similar word cluster.