Audio template matching - power correlation?
If you're cross correlating short time power spectra and disregard phase, then you're not really disregarding phase.

Phase of the global Fourier transform encodes the temporal structure, including the position of the transients, evolution of tones or the local incoherence of noise.

By capturing the time dependence in the moving window of the time-frequency power spectrum, the really important aspects of phase are represented in the temporal evolution of the frequency power density.

So what you are discarding is merely "local phase", which contains a lot of information that is however modified by even the most subtle processes like sound propagation, speaker reproduction, microphone recording, etc. These modifications don't affect the qualitative content of the sound a lot, and a robust sound recognition algorithm should be mostly insensitive to them.

That means discarding the local phase will make your recognition or correlation algorithm more robust and avoid misclassifications due to small inaudible phase errors, while at the same time preserving the total temporal structure of the signal.