After the Middle Ages, the castle gradually fell into ruin.
In 1927 the Duke of Beaufort gave the castle to Swansea Corporation; today, the castle is maintained under the responsibility of the City and County of Swansea council.