Alternatives to imputation of missing values? So I'm quickly learning that dealing with missing values for feature(s) in some of your observations is a part of every day life in data.  I get the gist of imputation, when/how it's appropriate and when it's not, and I'll read up on it in the near future.  But, what about this: 

Suppose you have predictors $X_1, \dots, X_p$, and you want to model, say, a binary response $Y$ via, say, logistic regression.  Instead of just imputing values to missing predictor values, couldn't you have a separate model built for every possible subset of the predictors, and apply that model for prediction when those predictors happen to be present?  Each of those models would naturally be trained on just the data for which those predictors (and also others) are present.  This seems to me a more reasonable approach than just making up values, but I have no theoretical justification for this.

I do realize that this involves building $2^p$ different models, with $2^p$ different model matrices, etc., but for a moderate $p$ and $n$ it could be feasible, and especially if only a few of your features tend to be missing more often than others.

Is this ever done?  And if so, is there a standard way to implement this in R?  In the case of logistic regression, you can specify to R's glm function how you want it to handle NA values, but your only options seem to be either tossing out observations altogether, or some kind of imputation scheme.

Thoughts?
Supposedly you had these models, divided into subspaces, then what if in your test dataset a point is close proximity to where the missing point is in your training data. What would be the best solution? Have the average output of the two closest models? Is the output binary? And then would you have to weight each model, based on that proximity? Then I am thinking that many submodels could overfit on the subset of the data. Just some thoughts, which may be challenging to overcome.

Moreover, you could use other types of Classifiers which deal with missing data, like some decision trees implementations.

Finally, data imputation is a delicate art, and doing it right will leverage your model. For instance, you wouldn't want to impute all the missing data with the average as then you may change your data distribution. Knn techniques or ensemble regressions would be preferred.