How can we use mean imputation without violating feature correlation? Mean imputation is generally bad practice because it doesn’t take into account feature correlation. Imagine we have a table showing age and fitness score and imagine that an eighty-year-old has a missing fitness score. If we took the average fitness score from an age range of 15 to 80, then the eighty-year-old will appear to have a much higher fitness score that he actually should. Thus, I wonder if we have any way to use mean imputation without violating feature correlation.
For me, a straightforward solution is to replace mean of the whole population in data by mean of a group of similar subjects. For example, in the above example, we can fill missing score of a person by mean score of other persons of the same age, or for more flexibility, in the same range of age. This approach still works if we have more than 2 features and still want to take into account correlation among all features. We just need to define a similarity metrics between subjects based on all features, e.g. cosine similarity. Of course, similarity computation is expensive, that is a limitation of this approach. So I wonder if there is any better alternative.
If you want to keep the feature correlation, use an imputer that solves for the missing values by using the feature correlation, such as KNN, regression, etc. These methods work well if there is no underlying reason that the data is missing - Missing at Random.