1. Weighted Loss in CrossEntropyLoss() 2. Combination of WeightedRandomSampler and subsampler
I wanted to implement class weights to my 3 class classification problem.
Tried by just directly adding the weights, which gives me an error when passing my model output and the labels to my loss
criterion = nn.CrossEntropyLoss(weight=torch.tensor([1,2,2]))

The error:
loss = criterion(out, labels)         
expected scalar type Float but found Long

So I print dtypes and change them to float but it still gives me the same error
labels = labels.float()
print(&quot;Labels Training&quot;, labels, labels.dtype)
print(&quot;Out Training &quot;, out, out.dtype)
loss = criterion(out, labels) 

&gt;&gt;Labels Training tensor([2.]) torch.float32
&gt;&gt;Out Training  tensor([[ 0.0540, -0.1439, -0.0070]], grad_fn=&lt;AddmmBackward0&gt;) torch.float32
&gt;&gt;expected scalar type Float but found Long

I also tried to change it to float64(), but it tells me that tensor Object has no attribute float64

Problem: I Havent tried this one out but I have seen that the more used approach would be the RandomWeightedSampler. My problem is that I use CV with K-Fold and use a SubSampler for that. Is it possible to use both? Havent foudn anything related to that.


For the first Problem, nn.CrossEntropyLoss requires the output to be of type float, label of type long, and weight of type float. Therefore, you should change the optional parameter of nn.CrossEntropyLoss &quot;weight&quot; to be float by:
criterion = nn.CrossEntropyLoss(weight=torch.tensor([1.0,2.0,2.0]))
loss = criterion(out, labels.long())

