title: Returning undistorted pixel coordinates rather than the image
body: <p>I need to undistort the pixel coordinates of an image -- and need the corrected coordinates returned. I do not want an undistorted image returned-- just the corrected coordinates of the pixels. The camera is calibrated, and I have the camera intrinsic parameters, and the distortion matrix. I am using OpenCV in python 3</p>
  
  <p>I have read up as much of the theory as I can find and questions here. Key info is:
  <a href="https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html" rel="nofollow noreferrer">https://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html</a></p>
  
  <p>This pretty clearly describes the radial distortion and tangential distortion that needs to be considered.
  radial:</p>
  
  <p>x_{corrected} = x( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)
  y_{corrected} = y( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)</p>
  
  <p>Tangential:</p>
  
  <p>x_{corrected} = x + [ 2p_1xy + p_2(r^2+2x^2)]
  y_{corrected} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]</p>
  
  <p>I suspect that I can't simply apply these corrections sequentially. Perhaps there is a function to do what I want to do directly, anyway -- and I'd love to hear about that.</p>
  
  <p>I can't simply use the normal undistort procedure on the image, as I am attempting to apply an IR camera's distortion correction to the depth data from the same camera. If you undistort a depth image like this -- you split pixels across coordinates and the answer makes no sense. Hopefully I am on the right track with this.</p>
  
  <p>The code so far:</p>
  
  <pre><code>import numpy as np
  import cv2
  imgIR = cv2.imread("/20190529-150017-305-1235-depth.png",0)
  #you could try this on any image...
  
  h,  w = imgIR.shape[:2]
  
  X = np.array([i for i in range(0,w)]*(h))
  X = X.reshape(h, w)
  
  Y = np.array([[i]*(w) for i in range(0,h)])
  
  fx =    483.0 #x focal length
  fy =    490.2
  CentreX = 361.4 #optical centre of the image - x axis
  CentreY = 275.6
  
  #Relative to the optical centre, it is possible to determine the `#coordinates of each pixel in the image` 
  #then do the above operation without loops using a scalar subtraction
  Xref = X - CentreX
  Yref = Y - CentreY
  
  #"scaling factor" refers to the relation between depth units and meters;
  scalingFactor = 18.0/36.0 # 18pixels / 36 mm;
  # I'm not sure what should be yet -- whether [pixels at the shelf]/mm
  #or mm/[pixels at the shelf]
  
  Z = imgIR / scalingFactor
  
  
  #using numpy
  Xcoord = np.multiply(Xref,Z/fx)
  Ycoord = np.multiply(Yref,Z/fy)
  
  #how to correct these coords for the radial and tangential distortion?
  #parameters as returned for the distortion matrix using 
  cv2.calibrateCamera
  dstvec = array([[-0.1225, -0.0159, 0.001616, -0.0018924,-0.00120696]]) 
  </code></pre>
  
  <p>What I am looking for is a new matrix of undistorted (radial and tangential distortion removed) X coordinates and a matrix of undistored Y coordinates -- with each matrix element representing one of the original pixels.</p>
  
  <p>Thanks for your help!</p>