Rhodes James was born in India as the third son of Colonel William and Violet Rhodes James.
Having begun his education in private schools in India, Rhodes James returned to England to attend Sedbergh School and then Worcester College at Oxford University.