Apply Principal Component Analysis (PCA) for RGB Images
General Idea

The general idea of Principal Component Analysis (PCA) is as following (Intuition over formalism):

Given a set of points in space (Inner Product Space) find a set of vectors (Directions) which are uncorrelated which span the data in the most energy preserving manner.

The tricky part is explaining "most energy preserving manner".
So we're talking on $ {L}_{2} $ norm and preserving means that if we ask for a given number of vectors, no other vectors will be able to approximate better the set in the $ {L}_{2} $ sense.

Working on Images

For a set of color images of size $ m \times n \times 3 $ we can work in either of the following approaches:


Vectorize image into single vector of size $ 3 m n $ and use regular PCA.
Use Tensor based approaches (Like the generalization of SVD to Tensors).


Task

The task is to build a dictionary for compression based on the PCA idea.
The data set for the task is based on the Utrecht ECVP face images data set.
The data set is composed of 131 images of size 900 x 1200 which were resized to 120 x 180.
In each approach the images were loaded image, converted to Float64 and scaled to range [0, 1]

Image Vectorization

Each images was vectorized to a single vector of length 64,800.
Taking all the images and removing the Mean Vector to create the data set.
The using SVD extracting the dictionary which spans the columns (Matrix $ U $ in the SVD).

Taking Image #1 in the data set with various numbers of components of the SVD:



The results are not as good as we usually get with Gray Scale images.
The reason that in the process the structure of image (3 Channels) is ignored and not leveraged.

So doing the same trick for each channels separately yield the following:



There is a small improvement (See the result for 75 components) but not significant.

Could we even do better?

Image as a Tensor

The trick here is we have many ways to project a tensor:


Project onto a tensor of a different space.
Project onto a vector.


Different approaches leads to different results as can be seen in A Survey of Multilinear Subspace Learning for Tensor Data:



MATLAB Code

The full code is available on my StackExchange Signal Processing Q58730 GitHub Repository.