Imrie was born in 1952 in Guildford, Surrey, the fourth of five children of Diana Elizabeth Blois (née Cator) and David Andrew Imrie, a radiologist.
Imrie was educated at Guildford High School, an independent school for girls in her hometown of Guildford, followed by the Guildford School of Acting.