Multi-label classification with nested features
I think it's always a good idea to start simple, so I'd simply suggest to try with all the features, including the different levels of &quot;nesting&quot; so around 2k apparently. Given that the dataset is large, I don't see any obstacle to trying this way. For the same reason I would start with a very simple model like Decision Trees or SVM, which have the additional advantage that they're fast to train. This could be a first step which provides you with a decent baseline, at least.
If the number of features turns is an issue for a more advanced option, I think this is a good case for using feature extraction (for example PCA): this would reduce the number of features and also merge features which represent the same information.