Early was born in the Red Valley section of Franklin County, Virginia, third of ten children of Ruth (née Hairston) and Joab Early.
He graduated from the United States Military Academy in 1837, ranked 18th of 50.