Following the war, in January 1866, Gillem was assigned command of the Fourth Military District, headquartered in Vicksburg, Mississippi and composed of the Federal occupation forces in Mississippi and Arkansas.
In 1875, Gillem became seriously ill and returned home to Tennessee to recuperate.