Principal component analysis is a technique that extract the best **orthogonal subspace** in which we can project our points with less information loss, maximizing the variance.

A linear auto encoder is a neural network composed by an encoder (single layer) that compresses our space in a new subspace, which **is not necessarily orthogonal**, and of a decoder that reconstruct our data with less information loss possible.

In substance, both the models are capable of features reduction, by projecting the original space in a new optimal subspace with and without a constraint of orthogonality.

In [this publication](https://towardsdatascience.com/understanding-variational-autoencoders-vaes-f70510919f73) in which is explained how *Variational Autoencoders* works, when PCA and linear autoencoder are compared, is stated that:

> 
> [...] Indeed, several basis can be chosen to describe the same optimal
>  subspace and, so, several encoder/decoder pairs can give the optimal
>  reconstruction error. Moreover, for linear autoencoders and contrarily
>  to PCA, **the new features we end up do not have to be independent (no
>  orthogonality constraints in the neural networks).** [...]
> 
> 
> 

Why if I project my points in a subspace that has no orthogonality constraints, **my features end up to be not necessarily independent**? And why in the orthogonal space the new features, linear composition of the previous one, are assumed to be independent? 

[![PCA vs linear Autoencoder](https://i.stack.imgur.com/c9jcL.png)](https://i.stack.imgur.com/c9jcL.png)