Hair emitted outside vertex group
Mr Zak has the right answer. The outer edge loop of my vertex group is weighted and allows some hairs to be emitted.

One workaround is to duplicate the faces in the mesh, assign them to a new vertex group and use that for the particle system: