Comparing spectrogram similarity over time
For spectrogram similarity, I suggest using a metric from image processing known as SSIM - Structural Similarity Index Metric. It is a quantity between $[0,1]$ which denotes the similarity between two images.
The paper is called Image Quality Assessment: From Error Visibility to Structural Similarity. The MATLAB function ssimval() will give you the metric.