Fox had begun living at the mansion in 1746, and in 1749 obtained a lease from Edwardes of the house and sixty-four acres of land for "99 years or three lives".
Fox died at Holland House in 1774 and thereafter it was inherited by his descendants.