Richmond was born in London, England, the son of Christopher Richmond, barrister and his wife, Maria Wilson.
Richmond returned to England in 1854 and married Mary Smith on 21 August 1856 before returning to New Zealand on the Kenilworth, which arrived in New Plymouth on 8 July 1857.