title: undistort vs. undistortPoints for feature matching of calibrated images
body: <p>I am trying to find the euclidean transform between two cameras (or actually, one moving camera) capturing the same scene, where calibration data <em>K</em> (intrinsic params) and <em>d</em> (distortion coefficients) is known.
  I am doing this via extraction of feature points, matching them and using the best matches as correspondences. </p>
  
  <p>Prior to resizing/feature detection/etc. I <code>undistort</code> both images</p>
  
  <pre><code>undistort(img_1, img_1_undist, K, d);
  undistort(img_2, img_2_undist, K, d);
  </code></pre>
  
  <p>where <code>img_.</code> are the inputs in <code>Mat</code> form obtained by <code>imread</code>. But really I only need the undistorted coordinates of the features I eventually use as correspondences, not those of all image pixels, so it would be more efficient, not to <code>undistort</code> the whole images, but only the keypoints. I thought I could do just that with <code>undistortPoints</code>, however the two approaches lead to different results.</p>
  
  <p>I resize the images</p>
  
  <pre><code> resize(img_1_undist, img_1_undist, Size(img_1_undist.cols / resize_factor,
              img_1_undist.rows / args.resize_factor));
   resize(img_2_undist, img_2_undist, Size(img_2_undist.cols / resize_factor,
              img_2_undist.rows / args.resize_factor));
   // scale matrix down according to changed resolution
   camera_matrix = camera_matrix / resize_factor;
   camera_matrix.at&lt;double&gt;(2,2) = 1;
  </code></pre>
  
  <p>After obtaining best matches in <code>matches</code>, I build <code>std::vector</code>s for the coordinates of said matches,</p>
  
  <pre><code>    // Convert correspondences to vectors
      vector&lt;Point2f&gt;imgpts1,imgpts2;
      cout &lt;&lt; "Number of matches " &lt;&lt; matches.size() &lt;&lt; endl;
      for(unsigned int i = 0; i &lt; matches.size(); i++)
      {
         imgpts1.push_back(KeyPoints_1[matches[i].queryIdx].pt);
         imgpts2.push_back(KeyPoints_2[matches[i].trainIdx].pt);
      }
  </code></pre>
  
  <p>Which I then use for finding the essential matrix.</p>
  
  <pre><code>    Mat mask; // inlier mask
      vector&lt;Point2f&gt; imgpts1_undist, imgpts2_undist;
      imgpts1_undist = imgpts1;
      imgpts2_undist = imgpts2;
      /* undistortPoints(imgpts1, imgpts1_undist, camera_matrix, dist_coefficients,Mat(),camera_matrix); // this doesn't work */
      /* undistortPoints(imgpts2, imgpts2_undist, camera_matrix, dist_coefficients,Mat(),camera_matrix); */
      Mat E = findEssentialMat(imgpts1_undist, imgpts2_undist, 1, Point2d(0,0), RANSAC, 0.999, 8, mask);
  </code></pre>
  
  <p>When I remove the calls to <code>undistort</code> and instead call <code>undistortPoints</code> on the keypoints, it does not produce the same result (which I would expect). The differences are sometimes minor, but always there.</p>
  
  <p>I read the documentation</p>
  
  <blockquote>
    <p>The function is similar to cv::undistort and cv::initUndistortRectifyMap but it operates on a sparse set of points instead of a raster image. </p>
  </blockquote>
  
  <p>such that the function should do what I expect.
  What am I doing wrong?</p>