Image similarity without perspective
Generative Adverserial Networks can be tolerant of this type of perturbation. In fact, if you look at figure 8 of the DCGAN paper, the authors demonstrate how the GAN learned to encode perspective directly such that the authors are able to use the GAN to rotate the camera on an image by applying a "turn vector".