Is there a significant population of neutron stars outside the galactic plane?
I'll partly answer your question with the following plot from the ATNF pulsar database (which is easy to play around with). The plot shows the Galactic z coordinate of the pulsars (distance from the Galactic plane in kpc) versus their estimated age (from their spin down rates, in years).

The basic picture is that young pulsars are all near the Galactic plane, because they are born from high mass stars that are born, live and die near the plane and even if the neutron star got a kick, there has been insufficient time to move far from the plane. A typical kick might be 500-1000 km/s, roughly 0.5-1 kpc in a million years.

Once pulsars reach a million years old, then indeed some reach a kpc above/below the Galactic plane and then a few even older examples reach 10 kpc.
Most normal pulsars only show pulsar activity for 10-100 million years (the older ones on the plot are probably a different population of pulsars in binaries), so they aren't observed at much greater distances (though there will be old neutron stars out there).