Can someone clarify what the linear assumption of PCA is? 0
For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some type of transformation if there is no linearity. Other articles state that your data has to be linearly separable. Which is it? Is it both?
Does it mean that that you first have to check if the independent variables are linear in relationship, then after applying PCA, check if the data is linearly separable?
OR
Check if the data, before applying PCA, is linearly separable with techniques like linear programming.
Then there is KERNEL PCA which after searching states that it is an extension of PCA where it is applied to nonlinear data. Does that mean nonlinear in relationship or linear inseparable?
PCA is the best (in the mean-squared error sense) linear decomposition method.

PCA is defined as an orthogonal linear transformation that transforms
the data to a new coordinate system such that the greatest variance by
some scalar projection of the data comes to lie on the first
coordinate (called the first principal component), the second greatest
variance on the second coordinate, and so on.
Wikipedia

The term &quot;Linear&quot; in PCA means:
a. That any data point is simply a linear combination of the principal components.
b. That the data matrix ($A$) can be decomposed via linear similarity transformations to diagonal matrix ($\Sigma$).
Ie
$$A = U \Sigma U^T$$
or
$$AU = U \Sigma$$
$\Sigma$ is the diagonal matrix of variances for each basis vector.
One can see at once that the linear algebra of the above formula makes clear the meaning of Linearity in PCA.
On the other hand decomposition methods like ICA (Independent Component Analysis) cannot be expressed via linear algebra as PCA above, since they require not only decorrelated components but independent components which is a stronger condition requiring non-linearities.
See also: https://datascience.stackexchange.com/a/80361/100269