The Harlyberg hill (256m/840 ft) north of the town was the site of a castle built in 1203 by the Welf king Otto IV of Germany to threaten the trade route to Goslar, as its citizens supported his Hohenstaufen rival Duke Philip of Swabia.
In 1174 the Counts of Wohldenberg established a Benedictine monastery at their ancestral seat west of Vienenburg, which converted into a Cistercian nunnery a few years later, confirmed by Emperor Frederick Barbarossa in 1188 and by Pope Honorius III in a 1216 deed.