Mathews was born in Augusta County, Virginia, on September 30, 1774, the son of a planter and Revolutionary War officer, George Mathews and his wife, Polly.
Mathews returned to Virginia for his education at Liberty Hall Academy (which later became Washington and Lee University).