Jacob Sharp was born in Pickensville, Alabama, to Elisha Hunter Sharp and his wife, Sallie (Carter) Hunt, who originally hailed from Hertford County, North Carolina.
As a young child, Sharp moved with this family to Lowndes County, Mississippi.