title: What is the Project Tango lens distortion model?
body: <p>The <a href="https://developers.google.com/project-tango/apis/c/struct_tango_camera_intrinsics" rel="noreferrer">Project Tango C API documentation</a> says that the <code>TANGO_CALIBRATION_POLYNOMIAL_3_PARAMETERS</code> lens distortion is modeled as:</p>
  
  <blockquote>
    <p>x_corr_px = x_px (1 + k1 * r2 + k2 * r4 + k3 * r6) y_corr_px = y_px (1
    + k1 * r2 + k2 * r4 + k3 * r6)</p>
  </blockquote>
  
  <p>That is, the <em>undistorted</em> coordinates are a power series function of the <em>distorted</em> coordinates. There is another definition in the <a href="https://developers.google.com/project-tango/apis/java/TangoCameraIntrinsics" rel="noreferrer">Java API</a>, but that description isn't detailed enough to tell which direction the function maps.</p>
  
  <p>I've had a lot of trouble getting things to register properly, and I suspect that the mapping may actually go in the opposite direction, i.e. the <em>distorted</em> coordinates are a power series of the <em>undistorted</em> coordinates. If the camera calibration was produced using OpenCV, then the cause of the problem may be that the OpenCV documentation contradicts itself. The easiest description to find and understand is the <a href="http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html?highlight=undistort" rel="noreferrer">OpenCV camera calibration tutorial</a>, which does agree with the Project Tango docs:</p>
  
  <blockquote>
    <p><img src="https://i.stack.imgur.com/8nn8k.png" alt="enter image description here"></p>
  </blockquote>
  
  <p>But on the other hand, the <a href="http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html#initundistortrectifymap" rel="noreferrer">OpenCV API documentation</a> specifies that the mapping goes the other way:</p>
  
  <blockquote>
    <p><img src="https://i.stack.imgur.com/8Meey.png" alt="enter image description here"></p>
  </blockquote>
  
  <p>My experiments with OpenCV show that its API documentation appears correct and the tutorial is wrong. A positive <code>k1</code> (with all other distortion parameters set to zero) means pincushion distortion, and a negative <code>k1</code> means barrel distortion. This matches what <a href="http://en.wikipedia.org/wiki/Distortion_%28optics%29#Software_correction" rel="noreferrer">Wikipedia says about the Brown-Conrady model</a> and will be opposite from the <a href="https://www.cs.auckland.ac.nz/courses/compsci773s1c/resources/A%20versatile%20camera%20calibration%20technique%20for%20high-accuracy%203D%20machine%20vision%20metrology%20using%20off-the-shelf%20TV%20cameras%20and%20lenses.pdf" rel="noreferrer">Tsai model</a>. Note that distortion can be <a href="http://people.csail.mit.edu/bkph/articles/Tsai_Revisited.pdf" rel="noreferrer">modeled either way</a> depending on what makes the math more convenient. I opened a <a href="http://code.opencv.org/issues/4297" rel="noreferrer">bug</a> against OpenCV for this mismatch.</p>
  
  <p>So my question: Is the Project Tango lens distortion model the same as the one implemented in OpenCV (documentation notwithstanding)?</p>
  
  <p>Here's an image I captured from the color camera (slight pincushioning is visible):</p>
  
  <p><img src="https://i.stack.imgur.com/D7FcN.jpg" alt="enter image description here"></p>
  
  <p>And here's the camera calibration reported by the Tango service:</p>
  
  <pre><code>distortion = {double[5]@3402}
  [0] = 0.23019999265670776
  [1] = -0.6723999977111816
  [2] = 0.6520439982414246
  [3] = 0.0
  [4] = 0.0
  calibrationType = 3
  cx = 638.603
  cy = 354.906
  fx = 1043.08
  fy = 1043.1
  cameraId = 0
  height = 720
  width = 1280
  </code></pre>
  
  <p>Here's how to undistort with OpenCV in python:</p>
  
  <pre><code>&gt;&gt;&gt; import cv2
  &gt;&gt;&gt; src = cv2.imread('tango00042.png')
  &gt;&gt;&gt; d = numpy.array([0.2302, -0.6724, 0, 0, 0.652044])
  &gt;&gt;&gt; m = numpy.array([[1043.08, 0, 638.603], [0, 1043.1, 354.906], [0, 0, 1]])
  &gt;&gt;&gt; h,w = src.shape[:2]
  &gt;&gt;&gt; mDst, roi = cv2.getOptimalNewCameraMatrix(m, d, (w,h), 1, (w,h))
  &gt;&gt;&gt; dst = cv2.undistort(src, m, d, None, mDst)
  &gt;&gt;&gt; cv2.imwrite('foo.png', dst)
  </code></pre>
  
  <p>And that produces this, which is maybe a bit overcorrected at the top edge but much better than my attempts with the reverse model:</p>
  
  <p><img src="https://i.stack.imgur.com/Wiru7.jpg" alt="enter image description here"></p>