 He needed a house fitting a man of his position, so construction of the castle on the site of the old manor house began in 1441.It was through his position as treasurer that he could afford the £3,800 construction cost of the original castle.The castle is not a defensive structure, but a palatial residence.