How does transformation matrix and camera matrix change when indexing pixels from -1 to 1 instead of 0 to N-1
Before attempting to answer this, let's first see what camera transformation matrix does. Consider the 3d point
$$ P_1 = [X_1, Y_1, Z_1]^T $$
Pre-multiplying by camera matrix gives
$$ K \cdot P_1 = 
\begin{bmatrix}
\frac{W/2}{tan(hfov/2)} &amp; 0 &amp; W/2 \\
0 &amp; \frac{H/2}{tan(vfov/2)} &amp; H/2 \\
0 &amp; 0 &amp; 1 \\
\end{bmatrix} 
\begin{bmatrix}
X_1 \\
Y_1 \\
Z_1
\end{bmatrix}
$$
$$ = 
\begin{bmatrix}
\frac{W/2}{tan(hfov/2)} X_1 + W/2 \\
\frac{H/2}{tan(vfov/2)} Y_1 + H/2 \\
Z_1
\end{bmatrix} $$
As we can see, $W/2$ at $K[1,1]$ scales $x$ value and the $W/2$ at $K[1,3]$ adds an offset. Thus, if $x \in [-1,1]$ then by this scaling and offsetting, we get $x \in [0,W]$.
Conversely, when $x \in [0,W]$, we should use the camera matrix as defined earlier. Hence when $x \in [-1,1]$, we shouldn't add the scaling or offsetting. Thus, only the camera matrix needs to be changed to
$$K = \begin{bmatrix}
\frac{1}{tan(hfov/2)} &amp; 0 &amp; 0 \\
0 &amp; \frac{1}{tan(vfov/2)} &amp; 0 \\
0 &amp; 0 &amp; 1 \\
\end{bmatrix}$$