By 1066 there was already a settlement at Acre, together with a church and the principal house of a substantial landowner called Toki, possibly on the site of the castle. As with most of his class, Toki was soon dispossessed. His lands were briefly held by a Flemish family ennobled by William the Conqueror and their Norfolk property descended to an heiress, Gundrada. It was through her that William I de Warenne, her husband, gained control of Castle Acre in about 1070.
Hamelin’s son and grandson were also prominent military and political figures, entertaining both Henry III (r.1216–72) and Edward I (r.1272–1307) at Castle Acre. His nephew, Richard Fitzalan, Earl of Arundel, whose descendants held Castle Acre until 1558, succeeded John, the 8th Earl and the last of the line, in 1347. By then, however, as indicated in a survey of 1397, the castle was little used and perhaps derelict.
In 1558 Thomas Howard, 4th Duke of Norfolk, sold the castle to Thomas Gresham, and then by his widow to Thomas Cecil (son of Elizabeth I’s lord high treasurer). In 1615 the outstanding lawyer and politician Sir Edward Coke, to whose descendants’ business the freehold belongs today, bought it.
Interestingly, given that the building had little practical use, Coke ordered some substantial repairs, including the ‘finishing up of eleven battlements and other masons’ work’. But from at least the later 17th century until the early 20th the site was used for grazing and the masonry was ruthlessly quarried for building and road making.