 Calhoun was the youngest of 12 children born to Sally Ann Calhoun and Morgan Calhoun.His father died when Calhoun was nine.At the age of 12, Calhoun attended a boarding school in Cherokee, North Carolina, where he learned the English language.