I am trying to reconstruct 3D-Coordinates from the 2D-Pixel-Coordinates in a Camera Picture using a side condition (in MatLab). I do have extrinsic and intrinsic camera parameters.

Using homogenous transformation I can transform 3D-Coordinates from a initial World Coordinate System to my Camera Corrdinate System. So here I have my extrinsic parameters in my Transform Matrix R\_world\_to\_Camera:

```
R_world_to_Camera = [ r_11, r_12, r_13, t1;
r_21, r_22, r_23, t2;
r_31, r_32, r_33, t3;
0, 0, 0, 1];

```

For intrinsic parameters I used Caltech's "Camera Calibration Toolbox for MatLab" and got these parameters:

```
Calibration results (with uncertainties): 

 Focal Length:          fc = [ 1017.21523   1012.54901 ] ± [ NaN   NaN ] 
 Principal point:       cc = [ 319.50000   239.50000 ] ± [ NaN   NaN ] 
 Skew:             alpha_c = [ 0.00000 ] ± [ NaN  ]   => angle of pixel axes = 90.00000 ± NaN degrees 
 Distortion:            kc = [ 0.00000   0.00000   0.00000   0.00000  0.00000 ] ± [ NaN   NaN   NaN   NaN    NaN ] 
 Pixel error:          err = [ 0.11596   0.14469 ] 

 Note: The numerical errors are approximately three times the standard deviations (for reference).

```

So I then get the Camera-Calibration-Matrix K (3x3) 

```
K = [1.017215234570303e+03, 0, 3.195000000000000e+02; 
0, 1.012549014668498e+03,2.395000000000000e+02; 
0, 0, 1.0000];

```

and using this I can calculate the 3D -> 2D - Projection-Matrix P (3x4) with:

```
P = K * [eye(3), zeros(3,1)];

```

When converting a Point in World-Coordinates [X, Y, Z]\_World I transform it first to Camera-Coordinates and then project it to 2D:

```
% Transformation
P_world = [X; Y; Z; 1]; % homogenous coordinates in World coordinate System
P_camera = R_world_to_Camera * [X; Y; Z; 1];

% Projection
P_pixels = P * camera;
P_pixels = P_pixels / P_pixels(3); % normalize coordinates

```

So my question now is how to reverse these steps? As side condition I want to set the Z-Coordinate to be known (zero in world-coordinates). I tried the solution proposed [here on Stackoverflow](https://stackoverflow.com/questions/7836134/get-3d-coord-from-2d-image-pixel-if-we-know-extrinsic-and-intrinsic-parameters), but somehow I get wrong coordinates. Any idea? Every help is appreciated a lot!!