Obtaining Correct (Least Squares Sense) Affine Transform Parameters Between Two Images
As a complement to the comments above, I am adding the slides for Image Registration by Leow Wee Kheng, where one can find:


  Given two images, how to register one with the other?
  
  
  Determine the corresponding points between the images. Manually mark corresponding points, or
  Detect and match features between views (see lecture on feature detection and matching).
  Determine the transformation between corresponding points. Assume that all pairs of corresponding points are related by the same
  transformation. Compute parameters of transformation given
  corresponding points.
  


Importance is laid on corresponding points. Indeed, even with an over-determined system, if the chosen pixels an weakly informative (on flat or noisy parts of the image), the matching might not be efficient. One can instead detect features (edges, corners, textural patches), and compute the registration based on them. 

Sometimes, one does not have the same points, or the same number of features between images. I recently have tested a efficient algorithm (CDP, or Coherent Point Drift) for non-standard chromatographic images (made of peaks or blobs), that can be tuned from rigid to smooth deformations:


Point-Set Registration: Coherent Point Drift, 2010
Our application, for the record: BARCHAN: Blob Alignment for Robust CHromatographic ANalysis, 2017