In supervised learning, are more data entries always better? I am doing a supervised learning problem and have 600,000 rows of data. I divided it into a training and test set and achieved a high accuracy that I was happy with. However, I had thrown away 300,000 entries as they contained significant missing data. When I redo the analysis except use mean, median, and mode imputation methods to fill in the missing data entries and repeat the training/testing - my accuracy drops by 4%.
Why is this? I thought more data would be better or at the least stay the same. Does this imply the imputation was inaccurate? Or perhaps I was biasing the sampling process by only selecting records with no missing entries the original time? How could I possibly know which case since I have no way of knowing how accurate the imputed values are with the true values that were missing?
That is a clue your data is missing-not-at-random.
The missing data is a result of a systemic process that also effects predicting target performance.