 He was sent to a prison camp in Richmond until he was exchanged (for James L. Kemper) on September 19, 1863.Upon his recovery, he was assigned by Major General Benjamin Butler to the command of a gunboat flotilla on the James River labeled the "Naval Brigade" and was attached to the XVIII Corps, Army of the James from April 28, 1864 to February 17, 1865.Graham led the Naval Brigade during the First Battle of Fort Fisher.