In 1135, a major Welsh revolt took place, however, and in response King Stephen restructured the landholdings along this section of the Marches, bringing together Skenfrith Castle and its sister fortifications of Grosmont and White Castle back under the control of the Crown to form a lordship known as the "Three Castles". Conflict with the Welsh continued, and following a period of détente under Henry II in the 1160s, the de Mortimer and de Braose Marcher families attacked their Welsh rivals during the 1170s, leading to a Welsh assault on nearby Abergavenny Castle in 1182. In response, the Crown readied the castle to face an attack, and in 1186, £43 was spent developing the defences followed by more work in 1190, probably establishing a stone keep and curtain wall. 13th–17th centuries In 1201, King John gave the Three Castles to Hubert de Burgh. Hubert was a minor landowner who had become John's household chamberlain when he was still a prince, and went on to become an increasingly powerful royal official once John had inherited the throne. Hubert began to upgrade his new castles, starting with Grosmont, but was captured while fighting in France. While Hubert was in captivity, King John took back the Three Castles and gave them to William de Braose, a rival of Hubert's. King John subsequently fell out with William and dispossessed him of his lands in 1207, but de Braose's son, also called William, took the opportunity presented by the First Barons' War to retake the castles. Once released, Hubert regained his grip on power, becoming the royal justiciar and being made the Earl of Kent, before finally recovering the Three Castles in 1219 during the reign of King Henry III. During Hubert's tenure, Skenfrith was entirely rebuilt; the old castle was levelled and a new rectangular castle with round towers and a central circular keep was constructed in its place. Hubert fell from power in 1232 and was stripped of the castles, which were placed under the command of Walerund Teutonicus, a royal servant; having been reconciled with the king in 1234, the castles were returned to him briefly but he fell out with King Henry III again in 1239 and they were taken back once again and assigned to Walerund. Walerund built a new chapel at the castle in 1244, and repaired the keep's roof. In 1254, Skenfrith Castle and its sister fortifications were