Modern methods for reducing dimensions and feature engineering
I'm not really knowledgeable about the modern techniques but I can tell you about the old ones: ;)
First there are two main approaches to dimensionality reduction: feature selection and feature extraction. You're using the former, which consists in discarding some of the original features. The latter consists in some kind of &quot;merging&quot; of similar variables, it can be worth trying especially if you have redundant features.
As you rightly noticed, feature selection based on individual features is rarely optimal. There are methods which can take the full set of features as a basis for selection, in particular genetic feature selection.