Sum of Squared Intensity Difference for block matching with ORB features, c++ I am trying to implement feature matching algorithm between image reference and image current.

The pipeline is as follows:


Gray scale Images (intensities = [0-255]) are imported.
Extract ORB keypoints in both reference and current images:


Ptr&lt;FeatureDetector&gt; detector = ORB::create();


Obtain blocks surrounding all keyPoints within specified window_size in both reference and current images. NOTE: If the blocks exceed the image upper or lower bounds, the block intensities would obviously be zero (Example: KP_A in image reference).




4.Instead of using existing feature matching algorithms in opencv, I am trying to utilize sum of squared of intensity differences (SSD) in the blocks acquired across ORB keypoints in reference and current images. 


Extracting correct features demands implementing crossCheckedMatching() to ensure features are chosen correctly.
Visualize matched features.


Complete code: here.

Problem 1: My final result looks something like this:


Whereas, I have read in few lecture notes and articles, SSD and block matching incorporated with cross checked matching method should provide decent matches. Does these matched features look any good in someone's eyes or awful after cross check matching procedure?

Problem 2: I do not know how to find the correct threshold for evaluating SSD values between two images (Empirical tuning vs. Existing logical approaches?).

Final goal, adapted from the textbook: 


use these matched features in RANSAC to filter outliers.
Find Homography (Theory says, 4 pairs of matched points needed!)
Camera Pose Estimation
3D reconstruction of environment


Apologies for such a long question, but I tried to squeeze it as much as I could.
Your results look reasonable. However, you're likely to get a somewhat unreliable result: the good matches are located in a small portion of the image, while a large spread would help in getting overall more accurate estimates (but that's probably due to your images).
Experimenting will work here! Otherwise, you can compute the expectation of the SSD between 2 blocks that are a rightful match under the noise level of your images. 


Eventually, bad matches should be eliminated by the RANSAC part, but you may have to do a trade-off between getting many but sometimes wrong matches (loose SSD threshold) or stricter ones (at the risk of not having enough matches o estimate the pose).