 Povlsen started playing football in local town Viby, but moved on to Danish top-flight club AGF Aarhus in 1984.In 1985, he was awarded the Danish 1st Division "Revelation of the Year".He moved abroad in 1986, when he signed for Spanish club Castilla CF, the reserve team of multiple European champions Real Madrid.