title: Python's libfreenect Kinect 1 Depth Map Data Format
body: <p>For a project, I have been attempting to transform the depth map given by libfreenect (a 480 by 640 matrix with depth values 0-255) into more usable (x,y,z) coordinates. </p>
  
  <p>I originally assumed that the depth variable <code>d</code> at each pixel represented the Euclidean distance between the sensor and point found. By representing the camera as a point, the matrix as a virtual image plane, and following the vectors from camera to pixels on the plane the distance <code>d</code>, I reconstructed what I thought were the actual coordinates. (Each point is located at distance <code>d</code> along the ray cast through the corresponding pixel).  As is evident below in Figure 1, the reconstructed room map (shown from above) is distorted.</p>
  
  <p><a href="https://i.stack.imgur.com/fjl4q.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fjl4q.png" alt="Euclidean Distance"></a> </p>
  
  <p><strong>Figure 1: <code>d</code> is Euclidean Distance</strong></p>
  
  <p>If I instead assume <code>d</code> represents the forward distance from camera to each point, the result is shown below, in Figure 2. Note the triangular shape, since measured points are located along rays projected from the robot's position. The x and y coordinates are of course scaled based on the depth, and z is the depth value <code>d</code>.</p>
  
  <p><a href="https://i.stack.imgur.com/fVdEv.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fVdEv.png" alt="Depth"></a></p>
  
  <p><strong>Figure 2: <code>d</code> is depth from camera, or z coordinate</strong></p>
  
  <p>For reference, here is the map generated if I do not scale the x and y coordinates by the depth, assume <code>d</code> is the z coordinate, and plot(x,y,z). Note the rectangular shape of the room map, since points are not assumed to be located along rays cast from the sensor.</p>
  
  <p><a href="https://i.stack.imgur.com/h6Q0l.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/h6Q0l.png" alt="Original"></a></p>
  
  <p><strong>Figure 3: Original Image</strong></p>
  
  <p>Based on the above images, it appears that either Figure 2 or 3 could be correct. Does anyone know what preprocessing libfreenect does on captured data points? I have looked online, but haven't found documentation regarding how depth is preprocessed before being stored in this matrix. Thanks for any help in advance, and I would be glad to supply any additional required information.</p>