What loss function should be used for sparse multilabel Image Classification pytorch
I solved it finally by taking the reciprocal of pos_weight instead of the one I mentioned in the question, like so:
pos_weight = (Number of positive examples)/(Number of negative examples).
Also used a little trick (by setting weight to 10000), to scale up the loss function, so that it does not show up as 0 (It has a tendency to do this)
To be honest I have no clue why this worked, and might be specific to my usecase, but hey atleast it works.
I'm working on a really sparse multilabel classification problem, with 1000 labels to make predictions on, and on average very few labels would be active. Around 5 would be the median positive labels per image.
I tried to use BCEWithLogitsLoss, but the results weren't great irrespective with different weights, however even with pos weight set as 10, 100, 300, 1000, 3000, the results were terrible, and the highest f1 score requires a high threshold, near 0.95, though while training the threshold is 0.5 post sigmoid activation. The highest f1 I was able to achieve is around 0.3, and the top 5 recall is around .22.
I also tried to use pos_weight = (Number of negative examples)/(Number of positive examples).
However none of these seem to work.
The issue is that the loss converges with these parameters, when there is still room for improvement.
I was wondering if there's more robust loss functions for such Imbalanced data. I was also looking at Focal Loss, NLL loss, Label Smoothing, but I don't know if they would work well for my problem. Any help would be much appreciated!