title: epipolar geometry / stereo vision using openCV
body: <p>I am developing epipolar geometry using openCV. 
  (the ultimate goal is estimating world coordinate of model or depth information from two images taken from different viewpoints using same camera)</p>
  
  <p>However i am struggling with estimating fundamental matrix and essential matrix that are used in epipolar geometry.</p>
  
  <p><img src="https://i.stack.imgur.com/7DYKq.jpg" alt="corresponding points with different viewpoint of camera"></p>
  
  <p>As shown above, I captured two images with different viewpoints (same camera). The same color points are corresponding points. </p>
  
  <p>the exact coordinates are ( i write it if you want to verify)</p>
  
  <p>points1 : left</p>
  
  <pre><code>points1.push_back(Point2f(157,223));
  points1.push_back(Point2f(190,312));
  points1.push_back(Point2f(157,541));
  points1.push_back(Point2f(136,443));
  points1.push_back(Point2f(355,374));
  points1.push_back(Point2f(346,406));
  points1.push_back(Point2f(252,410));
  points1.push_back(Point2f(254,379));
  </code></pre>
  
  <p>points2 : right</p>
  
  <pre><code>points2.push_back(Point2f(190,188)); 
  points2.push_back(Point2f(303,284)); 
  points2.push_back(Point2f(252,512)); 
  points2.push_back(Point2f(166,420)); 
  points2.push_back(Point2f(422,311));
  points2.push_back(Point2f(412,339));
  points2.push_back(Point2f(341,366));
  points2.push_back(Point2f(345,335));
  </code></pre>
  
  <p>using points, </p>
  
  <p>Mat FundamentalMatrix = findFundamentalMat(points1, points2, FM_RANSAC, 1.0, 0.99);</p>
  
  <p>openCV function is used to calculate fundamental matrix. however the calculated fundamental is strange I guess. </p>
  
  <p><img src="https://i.stack.imgur.com/dCgA2.png" alt="calculated fundamental matrices and constraints"></p>
  
  <ul>
  <li>left : using FM_RANSAC, right : using CV_FM_8POINTS</li>
  </ul>
  
  <p>the calculated fundamental matrix should satisfy x'T F x = 0</p>
  
  <p>where x' is (u2, v2, 1)^T and x is (u1, v1, 1) ^T</p>
  
  <p>with u1, v1 = points 1 coordinate and u2, v2 = points 2 coordinate.</p>
  
  <p>as illustrated above, i calculated the x'T F x = 0. however the third case (third corresponding points), it is not zero. </p>
  
  <p>of course i tried different method.</p>
  
  <pre><code>Mat FundamentalMatrix = findFundamentalMat(points1, points2, CV_FM_8POINT);
  </code></pre>
  
  <p>is also tried. </p>
  
  <p>however for this case, all of corresponding points do not satisfy constraint.</p>
  
  <p>whats the problem? is there any mistake i did? (what i am thinking is the 8 points are still insufficient, there should be more points)</p>
  
  <p>In addition, I estimated poses of the model (extrinsic matrices) for different viewpoints. and perform camera calibration. Thus E1 (extrinsic matrix for viewpoint 1) , E2, K (intrinsic matrix) are estimated. (I guess those are almost exact. because if i project a model point using E1, E2 and k, then it is projected at correct point on image).
  --> Then I estimate essential matrix E = K^T F K . 
  the E should satisfy determinant of E should 0 and has eigenvalue (1,1,0) or (sigma, sigma, 1)
  However it wasn't. </p>
  
  <p>What are the problems?</p>