 Talley was born on October 16, 1948, in Washington, D.C., as the son of Alma Ruth Davis and William C. Talley, a taxi driver.His parents left him with his grandmother, Binnie Francis Davis, who was a cleaning lady at Duke University, in Durham, North Carolina.Davis raised him and Talley claims he was given an "understanding of luxury."