How to define a "don't care" class in time series classification in Pytorch? This is a theoretical question.
Setup
I have a time series classification task in which I should output a classification of 3 classes for every time stamp t.
All data is labeled per frame.
The problem:
In the data set are more than 3 classes [which are also imbalanced].
My net should see all samples sequentially, because it uses that for historical information.
Thus, I can't just eliminate all irrelevant class samples at preprocessing time.
In case of a prediction on a frame which is labeled differently than those 3 classes, I don't care about the result.

My thoughts:

The net will predict for 3 classes
The net will only learn (pass backward gradient) for valid classes, and just not calculate loss for other classes.

Questions

Is this the way to go for &quot;don't care&quot; classes in classification?
How to calculate loss only for relevant classes in Pytorch?
Should I apply some normalization per batch, or change batch norm layers if dropping variable samples per batch?

I am using nn.CrossEntropyLoss() as my criterion, which has only mean or sum as reductions.
I need to mask the batch so that the reduction will only apply for samples whose label is valid.
I could use reduction='none' and do that manually, or I could do that before the loss and keep using reduction='mean'.
Is there some method to do this using built in Pytorth tools?
Maybe this can be done in the data-fetching phase somehow?

I am looking some standard, vanilla, thumb rule implementation to tackle this. The least fancy the better.

I am aware this is more than a single question.
They are still not separable, as the solution will be unified most likely.
Don't care classes are not advisable since don't care doesn't have any inherent pattern which would hurt your model inadvertently.
You can assign class weights for your classes. CrossEntropyLoss has an argument 'weights'.
BatchNormalization can be left alone since the normalization should happen on the basis of all data and not just some of the data.