Nullable one-hot encoding
The original case has categories: 0,1,2,3,4. To solve your question you can create the categories as follows:

0, 1, 2, 3, 4, 5
where the 5th category represents unsure nature of label, that is, its value will be 1 if you are unsure about the data point belonging to 0 or 4th label.

Now you will have one-hot encoding in the form of [0,0,0,0,0,1] for the data points where it might belong to 0th or 4th category. You can filter these rows out and perform training on the remaining rows.
Can I have a one-hot encoding like [none, 1, 0, 0, none];
i.e. I don't know if the sample class is 0 or 4, but I know its class 1, but not class 2 or 3
The loss function would then ignore where there's a none so we don't provide false-negatives.