 The exact date of the construction of the castle itself is uncertain, but following a French raid on Plymouth in August 1403, King Henry IV ordered the prior of Plympton and the abbot of Tavistock to further fortify the town with walls and towers.No finance seems to have been forthcoming from the Crown and Edmund Lacey, the Bishop of Exeter, later granted indulgences to those who made contributions towards the cost of the project.Instead of town walls, the defences took the form of a quadrangular castle with four towers.