He was born in the Parish of St George in the East, Stepney, London, the son of Victor Cavendish, 9th Duke of Devonshire, and his wife Evelyn Petty-Fitzmaurice.
He was, after his father's death, the owner of Chatsworth House, and one of the largest private landowners in both the United Kingdom and Ireland.