 Brown began his career as an apprentice at Bury, before moving to Huddersfield Town in 1977, where he helped the club to two promotions during Mick Buxton's reign as manager.He eventually left for Newcastle United in 1983, but missed the whole of the 1983–84 promotion season because of injury.He returned to Huddersfield in 1985 for another four seasons, playing more than 400 games for the club in all.