Pretrained features return worse results for sklearn classifier/pipeline
Normalization

Long story short normalization was the problem in my case. Because I haven't shared the transformer pipelines code in FeatureUnion, I've missed to tell that each one of them ends with normalization. So that's what the biggest difference.


In case with pretrained -> normalization is done over the whole dataset and nothing more 
In case of pipeline -> normalization is done 10 times (twice per each fold -> once for train data, once for test data) 


And that's how I do get different values => such difference in results.