P-channel MOSFET stays on when the gate driver is unpowered
Looking at your problem in detail, I think you should rework the circuit like this:



simulate this circuit &ndash; Schematic created using CircuitLab
AC coupling might be the simple solution to this.