What is the significance of phase margin in an open loop amplifier?
Can we just ignore the phase response and just focus on the gain and
bandwidth?

You can but, if the spectrum of the signal you are trying to amplify is wide AND the time-domain waveform shape is important to maintain, then you probably can't.

What is the significance of Phase Margin in open loop amplifier?

None.