Construction on the castle is thought to have begun at the end of the 12th century, when it was built as a sort of castellum.
The medieval castle was rebuilt in the end of the 16th century by Duke Charles (later Charles IX of Sweden) into a renaissance palace.