After the death of Queen Mary, Dormer's mother, Jane (née Newdigate), and Dormer's daughter, Jane, left England for the continent.
Dormer died at the age of 72, and was buried in the family vault in the church of Wing.