Camera calibration/pin hole camera model and working out 3d position
If you have the extrinsics then it is very easy. Having extrinsics is the same as having "camera pose" and the same as having the homography. Check this post in stackoverflow.

You have extrinsics, also called camera pose, which is described as a translation and a rotation:

$\displaystyle Pose =\begin{bmatrix}R|t \end{bmatrix} = \begin{bmatrix}R_{11} &amp;R_{12}&amp;R_{13}&amp;t_x\\R_{21}&amp;R_{22}&amp;R_{23}&amp;t_y\\R_{31}&amp;R_{32}&amp;R_{33}&amp;t_z \end{bmatrix} $

You can get Homography from Pose this way:

$\displaystyle   H = \frac{1}{t_z}\begin{bmatrix}{R_{1x}}&amp;{R_{2x}}&amp;{t_x}\\{R_{1y}}&amp;{R_{2y}}&amp;{t_y}\\{R_{1z}}&amp;{R_{2z}}&amp;{t_z}\end{bmatrix}$

Then you can project your 2D points into the corresponding 3D points by multiplying the Homography by the points:

$p_{2D}=\begin{bmatrix}x &amp;y &amp;1\end{bmatrix}\quad$ add $\quad z=1\quad$ to make them homogeneous

$p_{3D}=H*p_{2D}  $ 

$p= p / p(z)\quad$   Normalize the points