camera movement calculation
Is what I'm after even possible?


yes.

Let's start with the simplification your camera was a perfect pinhole camera.

For starters, imagine you pick an arbitrary point object in your scene. You then get a camera matrix $\mathbf C$ that allows your to predict the 2D position $\mathbf y$ in your image of your 3D-positioned object at $\mathbf x$:

$$
\mathbf y = \mathbf {Cx}, \quad \mathbf y= \begin{pmatrix} y_1\\y_2\\1 \end{pmatrix}\in\mathbb R^3,\,\mathbf x= \begin{pmatrix} x_1\\x_2\\x_3\\1 \end{pmatrix}\in\mathbb R^4,\,\mathbf C \in \mathbb R^{3\times 4}
$$

Notice that $\mathbf C$ isn't reversible, ie. there's multiple $\mathbf x$ that explain a $\mathbf y$.

Now, translating your camera by a movement vector $\mathbf m$ is the same as shifting the scenery by $\mathbf{-m}$, which leads to a $\tilde{\mathbf{x}}= \mathbf x - \mathbf m$.

Correspondingly, we get a

$$\tilde{\mathbf{y}} = \mathbf C\left(\mathbf x - \mathbf m\right)\text.$$

Now, since $\mathbf C$ is a matrix, i.e. a linear operator, this leads to 

$$\begin{align}
\tilde{\mathbf{y}} &amp;= \mathbf C\mathbf x - \mathbf C \mathbf m\\
 &amp;= \mathbf y -\mathbf C \mathbf m\quad \iff\\
\mathbf C\mathbf m &amp;= \mathbf y - \tilde{\mathbf y}\text.
\end{align}$$

As mentioned above, $\mathbf C$ isn't reversible, which means that from a single photo you can't tell the 3D coordinate of a point without additional info. However, if you can identify three  objects at different points $\mathbf y_{a,b,c}$ and $\tilde{\mathbf {y}}_{a,b,c}$, you should be able to elimate the unknowns from the system of equations.

Also note that your real camera isn't a perfect pinhole camera, and that might be used to actually get a less underdefined system, so you might be able to mathematically derive a unique solution with less than three points.