Algorithm to effectively find perspective transform in motion estimation
The suggested technique seems to be to gather points of interest, calculate a feature vector for them, and then find matching pairs of points in both images, followed by a calculation of the transform.

Here is a sample project: http://www.codeproject.com/Articles/95453/Automatic-Image-Stitching-with-Accord-NET

Here is a paper that claims improvement by preparing through a phase correlation stage: http://www3.nd.edu/~kwb/ThomasKareemBowyerIGARSS_2012.pdf