Reframing multilabel classification with imbalance in "both" directions Consider the multilabel problem when asking &quot;does the sample belong to this class&quot; with, for example, a movie label dataset where almost every movie is labelled &quot;drama&quot; because of this label's vagueness. It also has a few rare labels such as &quot;French late 60s noir&quot;.
To both boost recall, as well as maintain compatibility with most classifiers, we can both resample and train a binary classifier for each label individually (drama: yes/no, noir: yes/no).
But while &quot;French late 60s noir&quot; might be a 0.05:0.95 imbalance that requires serious effort for the model to guess, the &quot;drama&quot; category is so widely applied that the problem is reversed, i.e. 0.95:0.05.
How should one logically best proceed to maintain recall over precision, now that the &quot;target class&quot; is 95% of the labels (kind of breaks with the definition)? Upsample the non-target class? Downsample the target class? Reframe the problem as &quot;not drama&quot; to maintain underbalance for every label?
I don't think there's any obviously best option. I'd suggest trying a few reasonable solutions, evaluate on a development set and then pick the one which performs best. Don't forget to try with the original data as it is, resampling doesn't always work better.

Upsample the non-target class? Downsample the target class?

These two options are very likely to give the same result (assuming you use the same proportion).

Reframe the problem as &quot;not drama&quot; to maintain underbalance for every label?

For the classifier it would be exactly the same.