 During the reigns of Henry IV ( 1399 – 1413 ) and his successor Henry V ( 1413 – 1422 ) , Rochester Castle was in the guardianship of William , Earl of Arundel and his brother Richard . The castle was given to Henry V 's widow , Catherine of Valois , in 1423 as part of her dower to support her financially . She died in 1437 , at which point the castle came under the custodianship of the clerk of the King 's works . Despite this , there are no records of building work during the 15th century and almost nothing is known about Rochester Castle between then and the second half of the 16th century . The decline of the castle 's military significance is marked by the leasing of the surrounding ditch , beginning in 1564 at the latest . Between 1599 and 1601 stone from Rochester Castle was reused to build nearby Upnor Castle , an artillery fort . 
