 At the end of the 16th century, however, the Clifford family (Earls of Cumberland), became the hereditary constables of the castle, and Clifford's Tower took its name from the family at around this time.The deterioration of the castle continued into the reign of Elizabeth I, who was advised that it no longer had any military utility.Robert Redhead, the tower keeper, became infamous at the time for taking parts of the castle to pieces and selling off the stonework for his own profit.