Do I really need the N-MOS driver here?
The P-Channel MOSFET needs to switch up to 20V.


And that's the catch, when making the MCU output high impedance, you could have 20 V at that MCU's output. You would be violating the maximum ratings of the MCU.

But actually you would not get to 20 V at that output as the ESD protection diodes in the MCU will pull down that voltage (to roughly Vdd + one diode forward voltage or some safe clamping voltage, for example 6 V) so your PMOS will never be off.