How to mathematically explain the translational and rotational invariance of PCA
Let $e$ be the all-one column vector of length $n$.

Let $X\in \mathbb{R}^{n\times d}$ be the original matrix.

and $X'$ be the transformed matrix, that is we have $X'=XR^T+ev^T$.

Let's first compute the transformed mean assuming the mean of $X$ is $0$.

$$\frac{e^TX'}{n}=\frac{e^TXR'}{n}+\frac{e^Tev^T}{n}=v^T$$

To compute the singular vectors of the original matrix, theoretically, we can compute the eigenvalues of $$X^TX=UDU^T$$

And we can extract $W$ from $U$. 

To compute the PCA for the transformed data 

We have $$(X'-ev^T)^T(X'-ev^T)=(XR^T)^T(XR^T)=R^TX^TXR^T=RUDU^TR^T$$

Now, we can extract $W'$ from $RU$.

That is $$W'=RW.$$