How to estimate octave and size for visual features positioned at Harris corners I am currently working with and comparing the performance of several feature detectors provided by OpenCV as basis for visual feature matching.

I am using SIFT descriptors. I have accomplished satisfactory matching (after rejecting bad matches) when detecting MSER and DoG (SIFT) features.

Currently, I am testing my code with GFTT (Good Features to Track - Harris corners) to get a comparison, and also because in the final aplication, a set of GFTT features will be available from visual feature tracking process.

I am using cv::FeatureDetector::detect(...) which provides me with a std::vector&lt;cv::KeyPoint&gt; filled with detected features / keypoints / regions of interest. The structure cv::KeyPoint contains basic information about feature's location, as well as information about size and octave in which the keypoint has been detected.

My first results with GFTT were terrible until I compared the typical size and octave parameters in different types of features:


MSER sets the size (between 10 and 40px) and leaves the octave to 0
DoG (SIFT) sets both the size and the octave (size/octave ratio between 20 and 40)
GFTT the parameters are always: size = 3, octave = 0


I presume that is because the primary purpose of GFTT features was not to be used in matching but only in tracking. This explains the low quality of matching results, since the descriptors extracted from such tiny features stop being discriminatory and invariant to many things, including small, 1-pixel shifts.

If I manually set the size of GFTT to 10 - 12, I get good results, very similar to when using MSER or DoG (SIFT).

My question is: is there a better way to determine how much to increase the size (and/or octave) than just-go-with-10-see-if-it-works? I want to avoid hardcoding the size increase if possible and determine it programmatically, but hardcoding is okay as long as I have some solid arguments backing up my choices of the new size / size increase / size estimation algorithm.
I'm not sure there is actually a good response to your precise question : the scale-space thing of SIFT and SURF was actually developed to automatically estimate the "good" relevant neighborhood size around a corner-like keypoint (which is what good features to track are).

Now, more positive answers would be:


build a database of keypoints and good matches (e.g. using square calibration patterns) and create an automated performance assessment on this database to find the correct size. This task can actually really be automated (see the work by Mikolajczyk and Schmid about point matching evaluation)
embed your good features in an image pyramid to also have some kind of scale associated with them. You can look for references on multi-scale FAST and Harris interest points, that do something very similar to this procedure.


To heuristically find the maximum block size, you can compute estimates of your images with a box blur (which is more or less what the blockSize operator does) and see when the corner disappears. Note however that more blur takes the corner away from its true location. 

If you're really looking for some quick-and-dirty fix, try sizes between 5x5 and 11x11 (typical sizes used in stereo block matching). If you're looking for an intellectually satisfying criterion, then try to maximize the likelihood of good matching of two feature points under your noise level.