Quantify the difference between two audio spectrograms
You could do a cross correlation between signals to even out the time offset.
But why go for spectrogram in the first place? Do you have acoustic conditions varying over time? If not, go for the spectrum, should do the same job but without the time alignement problem.