How to deal with variable number of permutation invariant features?
Technically your problem is not about a variable number of features, since you can have a finite list of all the possible features. The standard case is just to use all these features, even if only a few of them are &quot;active&quot; for a particular instance (your first option). If the number of features is too high, then you need dimensionality reduction.
The advanced option is to use a feature extraction method: an unsupervised method which &quot;groups&quot; similar features together, the goal being to reduce dimensionality while preserving the information contained in the features. By definition this method modifies the set of features: the &quot;new features&quot; are not directly interpretable, as opposed to the original ones.
Note that the other way to reduce dimensionality is feature selection: these methods don't modify the features, but they discard the least informative ones. Feature selection is supervised (as opposed to feature extraction) because informativeness is measured with respect to the response variable.