How to warp a pixel patch to another camera I have two cameras with rotation and translation starting from the world origin and a patch in the first camera's reference coordinate system. I want to warp this patch into the second camera's reference coordinate system. My idea is to transform every pixel like this:

$$T=T_2-T_1$$

$$R=R_1^{T}R_2$$

$$x' = \left[(x  K_{inv} R) + T\right]\;K$$

Where $R$ and $T$ are the transformation matrices between the cameras and $K$ is the affine transformation matrix that describes the camera properties.

Is this correct? I know that when you go from 3D to 2D you need to project, but that matrix is not invertible, so you cannot really do it here? Am I missing something really big?
$x  K_{inv}$, with $x(3) = 1$, produces the pixels as 3D points "stuck on the screen" of the first camera. After transforming you get the same 3D points, but in the coordinate frame of the second camera, and then you project in that camera. So, overall, what you are doing is looking at the first camera from the point of view of the second one.

I suspect that is not what you want....