 Henry Tufton died in 1849 , and castle ownership fell to Hothfields . Maintenance was too expensive for the family , and by 1859 cattle were being kept in its gatehouse , and visitors complained that parts of the romantic ruin had become inaccessible . Without sufficient funds , the castle quickly fell into marked decay . 
