 James Richmond initially served as an orderly sergeant and was promoted to captain of Company A of the 15th Virginia Infantry.When that was consolidated with the 64th Virginia Mounted Infantryin late 1863 (where his brother William was a Captain), James Richmond received a promotion to major and later another promotion to lieutenant colonel.He served under Col. Auburn Pridemore (whom he later defeated to represent the same Congressional district).