 Francis Clifford died in 1641 , and the death of his son Henry Clifford , 5th Earl of Cumberland in 1643 left the line without a direct male heir . At this point , the Clifford estates reverted to Lady Anne . The English Civil War broke out in 1641 . Brougham was one of several castles in the generally Royalist Cumberland and Westmorland that were garrisoned by Cavalier forces . Sir John Lowther , the garrison commander , stated that he took control of Brougham Castle not because it was strategically important , but to deny the Parliamentarians of its use . Whilst under Royalist control , Lady Anne donated the income from her estates to the upkeep of her castles . In June 1648 , Appleby endured a four @-@ day siege before capitulating to the Parliamentarians , but lightly manned Brougham Castle succumbed easily to Colonel John Lambert . Although many castles in Cumberland and Westmorland were dismantled so they could not be used again , Brougham was spared this fate , most likely because it was not strategically important . In 1650 , Lady Anne Clifford began repaired Appleby and Brougham . Repairs were mostly complete by 1653 , but continued for several years afterwards , the work costing an estimated £ 40 @,@ 000 . By this time Brougham Castle was no longer a serious fortification and had become Anne 's country house . She laid out a garden on the site of the old Roman fort , which led to the discovery of such Roman artefacts as coins and three altars . A 10 @.@ 5 feet ( 3 @.@ 2 m ) stone wall was built around the garden , enclosing an area from the gatehouse to the south end of the Roman fort . 
