How does PCA work when we reduce the original space to 2 or higher-dimensional space? How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case.
$$\begin{array}{ll} \text{maximize} &amp; \mathrm{Tr}\left( \mathbf{w}^T\mathbf{X}\mathbf{X}^T\mathbf{w} \right)\\ \text{subject to} &amp; \mathbf{w}^T\mathbf{w} = 1\end{array}$$
You might want to have a look at the wikipedia article of PCA, where it says:
&quot;The $k$th component can be found by subtracting the first $k − 1$ principal components from $\mathbf{X}$:&quot;
$$\hat{\mathbf{X}}_k = \mathbf{X} - \sum_{s=1}^{k-1}\mathbf{X}\mathbf{w}_s\mathbf{w}_s^T$$
Then you repeat the process to find the next component:
$$\mathbf{w}_k = \arg\max \mathbf{w}^T\mathbf{\hat{X}}^T_k\mathbf{\hat{X}}_k\mathbf{w}$$
$$\text{s.t. } \mathbf{w}_k^T\mathbf{w}_k = 1$$