The house was largely rebuilt between 1778 and 1797 by a later Sir Gervase Clifton, who employed the premier architect in the north of England John Carr of York.
In 1896 Sir Hervey Juckes Lloyd Bruce, 4th Baronet (1843-1919) succeeded his cousin Henry Robert Clifton, to part of the Clifton estates.