Prediction for not completely well classified data I have a DataFrame of users, some of them are "bots" and they are identified with a bit equal to 1 in the "is_bot" column, if the bit is 0, the user is considered as "human".

The problem is that some users may be misclassified as "humans" instead of "bots" since the "bots" have been identified on the basis on an incomplete list in the gathering data phase.

I will train and test my model on this partially correct data, but when I test it, I will find that my model correctly predicts some users as "bots" even if in the original dataset they are "humans".

Correctly predicts means that, in reality, the users are bots because I checked some of them manually, but I can't do this for my entire dataset of 1 mln users.

This would result in a model with low accuracy, even if the predictions may be right.

How do I handle this problem?
You are dealing with noisy labels. I would not switch the labelling according to a trained model that learned on that particular data set, since probably you don't know which patterns lead to your models decision.

Otherwise if you know the reason for the wrong labelling, you could try to build methods yourself that run a sanity check on your data.

Nonetheless, models that don't overfit often can deal with a bit of noise in the labels. But you can also try methods that incorporate the noise aspect. Maybe check out https://stats.stackexchange.com/questions/218656/classification-with-noisy-labels