title: OpenCV Rectification with too much black areas on images C++
body: <p>I'm using OpenCV to calibrate and rectify a stereo system. I have a stereo camera with convergent eyes, actually I'm running these functions on this order:</p>
  
  <pre><code>for(int j=0; j &lt; ChessBoard.numSquares; j++)
      obj.push_back(Point3f((j/ChessBoard.numCornersHor)*ChessBoard.squareDim, (j%ChessBoard.numCornersHor)*ChessBoard.squareDim, 0.0f));
  [...]
  </code></pre>
  
  <p>Then I loop this for the numbers of the images that I would like to acquire</p>
  
  <pre><code>found_L = findChessboardCorners(image_L, ChessBoard.board_sz, corners_L, CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_NORMALIZE_IMAGE + CV_CALIB_CB_FILTER_QUADS + CALIB_CB_FAST_CHECK);
  found_R= findChessboardCorners(image_R, ChessBoard.board_sz, corners_R, CALIB_CB_ADAPTIVE_THRESH + CALIB_CB_NORMALIZE_IMAGE + CV_CALIB_CB_FILTER_QUADS + CALIB_CB_FAST_CHECK);
  found = found_L &amp;&amp; found_R;
  if(found)
    { 
      cornerSubPix(image_L, corners_L, Size(11, 11), Size(-1, -1), TermCriteria(CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1));
      cornerSubPix(image_R, corners_R, Size(11, 11), Size(-1, -1), TermCriteria(CV_TERMCRIT_EPS + CV_TERMCRIT_ITER, 30, 0.1));
      drawChessboardCorners(image_L, ChessBoard.board_sz, corners_L, found);
      drawChessboardCorners(image_R, ChessBoard.board_sz, corners_R, found);
  
      image_points[0].push_back(corners_L);
      image_points[1].push_back(corners_R);
      object_points.push_back(obj);
      printf("Right: coordinates stored\n");
      printf("Left: coordinates stored\n");
    }
  </code></pre>
  
  <p>After this block I call this two:</p>
  
  <pre><code>cameraMatrix[0] = Mat::eye(3, 3, CV_64F);
  cameraMatrix[1] = Mat::eye(3, 3, CV_64F);
  
  calibrateCamera(object_points, image_points[0], imageSize, cameraMatrix[0], distCoeffs[0], rvecs_L, tvecs_L);
  
  calibrateCamera(object_points, image_points[1], imageSize, cameraMatrix[1], distCoeffs[1], rvecs_R, tvecs_R);
  </code></pre>
  
  <p>and then:</p>
  
  <pre><code>rms = stereoCalibrate(object_points, image_points[0], image_points[1],
                      cameraMatrix[0], distCoeffs[0],
                      cameraMatrix[1], distCoeffs[1],
                      imageSize, R, T, E, F,
                      TermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, 1e-5),
                      CV_CALIB_FIX_ASPECT_RATIO+CV_CALIB_FIX_INTRINSIC);
  </code></pre>
  
  <p>finally:</p>
  
  <pre><code>stereoRectify(cameraMatrix[0], distCoeffs[0],
                    cameraMatrix[1], distCoeffs[1],
                    imageSize, R, T, R1, R2, P1, P2, Q,
                    CALIB_ZERO_DISPARITY, -1, imageSize, &amp;roi1, &amp;roi2 );
  
  initUndistortRectifyMap(cameraMatrix[0], distCoeffs[0], R1, P1, imageSize, CV_16SC2, map11, map12);
  initUndistortRectifyMap(cameraMatrix[1], distCoeffs[1], R2, P2, imageSize, CV_16SC2, map21, map22);
  remap(imgL, imgL, map11, map12, INTER_LINEAR,BORDER_CONSTANT, Scalar());
  remap(imgR, imgR, map21, map22, INTER_LINEAR,BORDER_CONSTANT, Scalar());
  </code></pre>
  
  <p>This is basically what I'm doing, but the result is really bad, because the images have a really big black area. Here is an example:</p>
  
  <p><img src="https://i.imgur.com/EoaCIHb.png?1" alt="Left From OpenCV"></p>
  
  <p>and this is the rectify image that I have to obtain, that is taken rectified directly from the camera:</p>
  
  <p><img src="https://i.imgur.com/QSXcQrH.png?1" alt="Left From Camera"></p>
  
  <p>As you can see it seems that the image is translated on the right side and cut, with the right one is the same but is translated on the left side and the result is nearly the same.</p>
  
  <p>So how can I achieve a better result similar to the last one? Where is the problem? 
  As an additional data  I have noticed that rms is not so good, about 0.4, and the re-projection error is about 0.2, I know that they must me a bit lower, but I have try many many times with different pattern, illumination and so on, in calibration but I always take the same result or even worst.</p>