Principal Component Analysis (PCA) on Convolutional Neural Network (CNN) Features
I'm not into details of this specific case but I can see some logic.

A convolution layer can be reformulated as a Matrix Multiplication:

$$ y = W x $$

Let's say we trained on Data Set $ {x}^{1} $ which is big and general.
Namely we expect the trained weights $ {W}^{1} $ to be good enough for almost any other data set.  

Let's assume we have another data set $ {x}_{2} $ with its matching weights $ {W}^{2} $.
By our assumption about how good $ {x}^{1} $ generalizes it means $ {W}^{1} \sim {W}^{2} $ with regard to some metric.  

If this metric is the Frobenious Norm / Matrix Induced $ {L}_{2} $ Norm then using PCA (Which is basically SVD) makes sense as using the PCA we take the component of $ {W}^{1} $ which generalizes the most and use it for other case.