With his state's secession from the Union, Isaac returned to Burke County, and with his brother Alphonso, recruited Company E of the 6th North Carolina Regiment.
With Hoke's wounding at the Battle of Chancellorsville in May 1863, Avery temporarily assumed command of the brigade in time for the Gettysburg Campaign.