Until the mid-16th century it remained a defensive castle; there is a legend of a tunnel under the Isar connecting it with the Neuburg on the Brauneck. From 1570 on, rebuilding changed its appearance and gave it characteristics of an aristocratic residence.