Why training error is larger than validation error after PCA? We have 4000 features and we are applying Principal Component Analysis to reduce them a small number of features from 20 to 100.

We are performing linear regression.

Both training and validation errors are worse than without any use of PCA.

However we notice that the training error is not lower and actually has become slightly larger than the validation error.

Is this something expected?



Edit:

The variance ratio of the first principal components is high enough in comparison to the others.

What we are noticing is that if you use all of the 4000 principal components then the gap between training error and validation is larger and as you repeat the process and you reduce the principal components to 1000, 200, 100, 10, etc. then the training error comes closer to the validation error (even though both are being increased as expected)
The basic principle that helps us implement PCA is it's ability to explain the variation, so it depends how much PC you are taking and how much variation they are explaining. They are definitely not explaining 100% of variation so there are good chance that Validation Error is higher.

Before applying PCA you need to understand why we apply it? We apply it to reduce computational effort and practical implementation of problem. The idea of PCA is not to give better result than actual variables in problem.

Better results are generally available with complete list of variables.