Are the compact objects in supernova remnants isolated neutron stars?
There are plenty of hard-x-ray binaries that involve a compact object with a companion.  Some should be in supernova remnants, but I don't know how many.  Are you sure there is a problem?