How to compute 2D displacement vector for binary image registration? My inputs are several binary images like:

They have globally the same content but may be unregistered as they are produced from a hand camera. What I would like to compute is the 2D displacement vector from the first image to the second. I use OpenCV and my first attempt was to compute feature points (SURF algorithm) and the affine transform between both images. But of course, feature description are kinda poor on a binary image, therefore the matching is difficult and the matching map very inaccurate.

Does anyone has an idea how I could do this?
I am somewhat surprised that feature points don't work that well. I have had success registering shapes like yours using either


Harris points, this is a corner detector, in combination with the RANSAC algorithm. See the wiki or Peter Kovesi his site
Using a feature detector like SURF or SIFT in combination with an edge map of the image prior to feature detection followed by some form of robust matching. 


EDIT

I gave it a go with matlab and tried some variations on this theme. Currently I use


SIFT features from the edge map to ascertain rotation and scaling, although they are very small between the images you provided
RANSAC for robust matching
Cross correlation to ascertain the translation between the two images






Source on github. 

An alternative for RANSAC could be the Hough transform voting/binning approach that was proposed by the inventor of SIFT, David Lowe.