HC-05 and p-channel mosfet
That's not a good choice of MOSFET I'm afraid.

The on resistance is quoted as 14Ω when the gate is at -10V - however, you are only able to get -5V, so it will be considerably worse.

That massive on resistance will be causing a massive voltage drop, which is what you are experiencing.

You should find a MOSFET with an on resistance that is in the milli-ohm range when the gate is between 0V and -5V.