title: Results of SURF keypoint dection and FLANN matching between images vastly different for C++ and Python OpenCV libraries
body: <p>I've implemented a workflow of matching keypoints between images in Python, and I'm in the process of converting it to Objective C++, for use in an iOS project. I'm using OpenCV 3.4 on both platforms. Though I'm matching my code, the results seem vastly different.</p>
  
  <p>Python implementation:</p>
  
  <pre><code>def simple(source_image_name, sample_image_name):
      surf = cv2.xfeatures2d.SURF_create(400, nOctaves=3, nOctaveLayers=4, extended=False, upright=True)
  
      source_image = cv2.imread(source_image_name, 0)
  
      # Detect keypoints in source image
      source_keypoints, source_descriptors = surf.detectAndCompute(source_image, None)
  
      print("source keypoints: " + str(len(source_keypoints)))
  
      sample_image = cv2.imread(sample_image_name, 0)
  
      # Detect keypoints in sample image
      sample_keypoints, sample_descriptors = surf.detectAndCompute(sample_image, None)
  
      print("sample keypoints: " + str(len(sample_keypoints)))
  
      # Find matches between the images
      FLANN_INDEX_KDTREE = 0
      index_params = dict(algorithm=FLANN_INDEX_KDTREE, trees=5)
      search_params = dict()
      flann = cv2.FlannBasedMatcher(index_params, search_params)
  
      matches = flann.knnMatch(source_descriptors, sample_descriptors, k=2)
  
      print("matches: " + str(len(matches)))
  
      # Filter to good matches
      good_matches = []
  
      for match1, match2 in matches:
          if match1.distance &lt; match2.distance * 0.75:
              good_matches.append(match1)
  
      print("good matches: " + str(len(good_matches)))
  </code></pre>
  
  <p>Objective C++ implementation:</p>
  
  <pre><code>+ (void)simpleWithSourceImage:(UIImage *)sourceImage sampleImage:(UIImage *)sampleImage {
      Ptr&lt;SURF&gt; surf = cv::xfeatures2d::SURF::create(400, 3, 4, false, true);
  
      cv::Mat sourceImageMat;
      UIImageToMat(sourceImage, sourceImageMat);
  
      std::vector&lt;KeyPoint&gt; sourceKeypoints;
      cv::Mat sourceDescriptors;
  
      //Detect keypoints in source image
      surf-&gt;detectAndCompute(sourceImageMat, noArray(), sourceKeypoints, sourceDescriptors);
  
      NSLog(@"source keypoints: %lu", sourceKeypoints.size());
  
      cv::Mat sampleImageMat;
      UIImageToMat(sampleImage, sampleImageMat);
  
      std::vector&lt;KeyPoint&gt; sampleKeypoints;
      cv::Mat sampleDescriptors;
  
      //Detect keypoints in sample image
      surf-&gt;detectAndCompute(sampleImageMat, noArray(), sampleKeypoints, sampleDescriptors);
  
      NSLog(@"sample keypoints: %lu", sampleKeypoints.size());
  
      //Find matches between the images
      FlannBasedMatcher matcher(new cv::flann::KDTreeIndexParams(5));
  
      std::vector&lt;std::vector&lt;DMatch&gt;&gt; matches;
  
      matcher.knnMatch(sampleDescriptors, sourceDescriptors, matches, 2);
  
      NSLog(@"matches: %lu", matches.size());
  
      //Filter to good matches
      std::vector&lt;DMatch&gt; goodMatches;
  
      for (int i = 0; i &lt; matches.size(); i++) {
          std::vector&lt;DMatch&gt; match = matches[i];
  
          if (match[0].distance &lt; match[1].distance * 0.75) {
              DMatch goodMatch = match[0];
  
              goodMatches.push_back(goodMatch);
          }
      }
  
      NSLog(@"good matches: %lu", goodMatches.size());
  }
  </code></pre>
  
  <p>Both of these implementations:</p>
  
  <ol>
  <li>Load the images into memory, in the format they need them in.</li>
  <li>Detect keypoints and descriptors on each image.</li>
  <li>Look for matches between the images.</li>
  </ol>
  
  <p>Python console logs:</p>
  
  <pre><code>source keypoints: 16133
  sample keypoints: 18265
  matches: 16133
  good matches: 3785
  </code></pre>
  
  <p>C++ console logs:</p>
  
  <pre><code>source keypoints: 16281
  sample keypoints: 18040
  matches: 18040
  good matches: 558
  </code></pre>
  
  <p>Though they've been given the same images, the number of detected keypoints are slightly different across both images. Perhaps this is to do with the way it's reading the images in. The slight difference there of about 1% isn't too much of a concern.</p>
  
  <p>The matches however, have more of a disparity, with about a 11% difference, though it's actually higher for C++. And then the number of good matches is <em>far</em> lower for C++ - a 86% difference.</p>
  
  <p>I looked into the <code>matches</code> difference a bit more...</p>
  
  <p>Python matches:</p>
  
  <pre><code>match 1 average: 0.220831272634
  distance average: 0.840705045962
  </code></pre>
  
  <p>C++ matches:</p>
  
  <pre><code>match 1 average: 0.298346
  distance average: 0.924873
  </code></pre>
  
  <p>So the match 1 average is notably higher for the C++ results, and the average distance between the first match and second match is quite a lot closer.</p>
  
  <p>I tried changing that initial code - for loading the image files, to this:</p>
  
  <pre><code>NSFileHandle* sourceHandle = [NSFileHandle fileHandleForReadingAtPath:sourceImageName];
  NSData *sourceData = [sourceHandle readDataToEndOfFile];
  v::Mat sourceImageMat = cv::imdecode(cv::Mat(1, [sourceData length], CV_8UC1, (void*)sourceData.bytes), CV_LOAD_IMAGE_UNCHANGED);
  </code></pre>
  
  <p>That <em>did</em> affect the results, but didn't impact the conclusion - C++ matches are still far lower, with a high distance average, and a close distance match between results:</p>
  
  <pre><code>source keypoints: 16116
  sample keypoints: 18253
  matches: 18253
  good matches: 518
  match 1 average: 0.299299
  distance average: 0.925706
  </code></pre>
  
  <p>I also tried looking at the match 2 average on the Python code, and the average was 0.26 - still lower than the math 1 average on the C++ code. In order words, the 2nd best results for Python were on average better than the best matches for C++.</p>
  
  <p>I'm trying to figure out where this disparity is coming from. Are my implementations not matching?</p>