Data Cleaning Techniques - What to drop? I have been struggling with this problem for a few weeks but it seems to be out of my league. I have a dataset of product sales over the course of 55 weeks. It contains information on the store ID, and the product ID. However, some of the data is missing. Some store IDs do not have continuous data for the 55 weeks, and some item IDs do not have continuous data. Also, combinations of store IDs and item IDs have missing data points. What is the most efficient way to decide what to drop and what to keep for the best results in my model? If the product I am forecasting is continuous that is a good start, but I am not sure what the best combination of stores and items to drop is.
Check the percantege of missing values in each column. If it is high then you should remove this column, otherwise you are able to try to impute the missing values. For the latter you may replace them with the mean (median or mode etc) or try to predict them using the other known values. You should check if these are values are mising at random or not.