Subsequent to heading the Academy, Fitch served briefly in the Office of the Undersecretary of the Navy before becoming the senior member of the Naval Clemency and Prison Inspection Board in March 1947.
Admiral Fitch died due to a heart ailment combined with a bout with pneumonia in Newcastle, Maine his adopted state, on May 22, 1978 shortly before his 95th birthday.