How to improve a regression prediction model for data with parameters which are out of sample range?
I have encountered case in which potent tree learners acted like Nearest Neighbor variants: They would learn to divide the search space so that only examples remained that where close on some meaningful axis (in my case lat and long :)). This could still generalize to examples in the test set that shared those features but not to genuinely unseen examples. Or to put it in another way: The method is potent enough to find rules that work well, but do not generalize in the way you want it to (aka over fitting, but the maybe in a specific way). What helped me were two things: First of all I tested this characteristic of the domain by just using KNN,only feeding it subsets of the feature space (and sure enough KNN over lat, lng worked like a charm). This helped me understand that examples that were in the same space time coords shared certain history, so the second action was to eliminate those examples in the train space and evaluate on an a set of examples totally outside of the coordinates (space and time) of the train set. It is hard to say where this behavior could lie in your case without having a peek at the code, but perhaps my anecdote helps.