Correlation between slightly misaligned images I am matching camera images of projection screens with rendered pages of PDF documents using Pearson's and Spearman's correlation coefficient. When the images (3, 4, 5) are aligned, this works fine in spite of the clear shortcomings (optical aberration in the camera images, the random sample is not independent, uneven illumination, etc.).

In the figure below, Pearson's r between the left and middle images is 0.8395 and Spearman's r is 0.6499. Pearson's r between the left and right images is 0.7139 and Spearman's r is 0.5066.




However, when the content of the rendered PDF page differs slightly from the document in the camera image or when the images (8, 9, 10) are more than slightly misaligned, the correlation coefficient ceases to be useful.

In the figure below, Pearson's r between the left and middle images is 0.771 and Spearman's r is 0.4719. Pearson's r between the left and right images is 0.779 and Spearman's r is 0.4963.




Reducing the images to density estimates (i.e. discarding the pixel positions and comparing histograms) would likely remove valuable information. Non-feature-based image registration seems like a more promising direction. What technique would you recommend to attack this problem?
As suggested by Dan Boschen, I maximized the Spearman's correlation coefficient over all possible “small” translations – where small is less than 15 pixels in L1-distance – of the rendered PDF page images. The results were underwhelming, but I suppose this method would work much better if the camera images weren't optically distorted.

In the figure below, Pearson's r between the left and middle images is 0.789 and Spearman's r is 0.479. Pearson's r between the left and right images is 0.8008 and Spearman's r is 0.5094.



As suggested by Marcus Müller, I used a feature detector (ORB, not SURF) to find keypoints in the images. Rather than directly use the feature descriptors for matching, I used them to find a homography that registers the two images.

In the figure below, Pearson's r between the left and middle images is 0.6244 and Spearman's r is 0.4921. Pearson's r between the left and right images is 0.0213 and Spearman's r is 0.036. For my test case, this works remarkably well.