Ideally I'd like a reference of the math involved, so I can start breaking down the problem.

The simplest thing I can deduce about this problem is that the part of the sphere that the user is currently seeing is a circle on the image that is located somewhere on the image, and the rest of the coordinates on the image are somehow mapped to 3D with a linear transformation.

Consider the following example:

![enter image description here](https://i.stack.imgur.com/SgfWF.png)

If I rotate the sphere theta degrees on the Y axis (up/down) to the left, the sun will come close to the center of the sphere and the tree that is currently positioned in the center will be shifted to the left as the three on the left currently is. You can consider that the volume of the image is exactly enough to have the image completely wrap the sphere.