Phase detection with non-ideal PFD
I'm not going to work through the details of your specific schematic. I just want to make a few general statements:


As long as the transfer function of your detector is monotonic, it can be used to drive the frequency error of your PLL to zero. The input-to-output delay of the detector doesn't affect this.
As long as the delays from the two inputs to the output are matched, you can also drive the phase error to zero. If not, you're left with a small residual constant phase error, which can usually be compensated for by other means.