Semantic Distance Measure Between Images
One of the most known image similarity metric is the SSIM. You can take a look to the next links:

https://en.wikipedia.org/wiki/Structural_similarity
https://www.imatest.com/docs/ssim/
https://www.cns.nyu.edu/~lcv/ssim/

EDIT 1:
Okay, you refer to semantic similarity. I have not worked on that issue really, and probably there are new DNN solutions to that purpose. I have not checked the state-of-the-art related to this topic.
Even so, I have made a quick test using the skimage SSIM and the difference between images to give you some idea about what happens with these metrics on several images. Find in the next images some hints about their performance for your target application:


Please, notice the values of the SSIM, MSE and STD DIFF on the images. SSIM and MSE works very well to compare quality of the same image. But not for semantic comparison.
Cheers.