title: Kinect mapping color image to depth image in MATLAB
body: <p>I have collected data using Kinect v2 sensor and I have a depth map together with its corresponding RGB image. I also calibrated the sensor and obtained the rotation and translation matrix between the Depth camera and RGB camera.</p>
  
  <p>So I was able to reproject the depth values on the RGB image and they match. However, since the RGB image and the depth image are of different resolutions, there are a lot of holes in the resulting image.</p>
  
  <p>So I am trying to move the other way, i.e. mapping the color onto the depth instead of depth to color.</p>
  
  <p>So the first problem I am having is that the RGB image has 3 layers and I have to convert the RGB image to grayscale to do it and I am not getting the correct results.</p>
  
  <p>Can this be done?</p>
  
  <p>Has anyone tried this before?</p>