In the summer of 1863, Reilly's 104th OVI was assigned to Maj. Gen. Ambrose Burnside's Army of the Ohio during its campaign in East Tennessee, including operations around the Cumberland Gap. They marched to Knoxville and joined the XXIII Corps. In the autumn of 1863, Reilly commanded a brigade of infantry in the XXIII Corps during the Siege of Knoxville. The following year, he led his brigade with competence during the Atlanta Campaign.