title: Undistort a Fisheye Image using cv::fisheye::undistortImage
body: <p>I am trying to undistort a fisheye image using OpenCV. I obtained the camera matrix and the distortion coefficients from the internal memory of the camera. I'm assuming them to be accurate.  As you can see in the below code, I'm using <a href="https://docs.opencv.org/4.1.2/db/d58/group__calib3d__fisheye.html#ga167df4b00a6fd55287ba829fbf9913b9" rel="nofollow noreferrer">cv::fisheye::undistortImage</a>. I found this <a href="https://github.com/IntelRealSense/librealsense/issues/3880#issuecomment-489049908" rel="nofollow noreferrer">GitHub Issue post</a> claiming that it should work, however, the undistorted image frame doesn't look right! So, clearly something isn't working.</p>
  <p>Here is the code:</p>
  <pre><code>#include &lt;opencv2/highgui.hpp&gt;
  #include &lt;opencv2/calib3d.hpp&gt;
  #include &lt;opencv2/core/mat.hpp&gt;
  
  int main()
  {
      cv::Mat cameraMatrix = cv::Mat(3,3, CV_64F, double(0));
      cv::Mat distortionCoeffs = cv::Mat(1,4, CV_64F, double(0));
  
      cameraMatrix.at&lt;double&gt;(0, 0) = 286.7037963867188;
      cameraMatrix.at&lt;double&gt;(0, 1) = 0;
      cameraMatrix.at&lt;double&gt;(0, 2) = 413.3463134765625;
      cameraMatrix.at&lt;double&gt;(1, 0) = 0;
      cameraMatrix.at&lt;double&gt;(1, 1) = 286.7817993164062;
      cameraMatrix.at&lt;double&gt;(1, 2) = 397.1785888671875;
      cameraMatrix.at&lt;double&gt;(2, 0) = 0;
      cameraMatrix.at&lt;double&gt;(2, 1) = 0;
      cameraMatrix.at&lt;double&gt;(2, 2) = 1;
  
      distortionCoeffs.at&lt;double&gt;(0,0) = -0.01078350003808737;
      distortionCoeffs.at&lt;double&gt;(0,1) = 0.04842806980013847;
      distortionCoeffs.at&lt;double&gt;(0,2) = -0.04542399942874908;
      distortionCoeffs.at&lt;double&gt;(0,3) = 0.008737384341657162;
  
      cv::Mat input_frame = cv::imread(&quot;fisheye_input.png&quot;);
      cv::Mat output_frame;
  
      cv::fisheye::undistortImage(input_frame,output_frame,cameraMatrix,distortionCoeffs, cv::noArray(), cv::Size(input_frame.cols,input_frame.rows));    
  
      cv::imshow(&quot;Input Image&quot;, input_frame);
      cv::imshow(&quot;Output Image&quot;, output_frame);
      cv::waitKey(-1);
      return 0;
  }
  </code></pre>
  <p>Code output:</p>
  <p><a href="https://i.stack.imgur.com/2Y1nm.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2Y1nm.png" alt="enter image description here" /></a></p>
  <p>In case you want to try it yourself here is the original fisheye image:</p>
  <p><a href="https://i.stack.imgur.com/hSA6q.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hSA6q.png" alt="enter image description here" /></a></p>