XGBoost unreasonable splitting value in nodes
A possible explanation: you have all non-missing values going to the 'text < 4' branch, and all missing values to the other - 'text > 4' - branch. Can you verify?
My xgboost model trained for a regression task in python using the xgboost package version 0.6 is using strange values for splits. Some values used as a splitting criteria are not present in the training dataset at all.
Example:
- there's a variable 'text' with values in the train set of [Missing,1,2]
- yet, a derived splitting criteria of a node in the trained model is 'text < 4'
What could be a possible reason of such a split when no such value (-> 4) can be found in the data set?The split does not increase the information gain, since all samples follow one branch after this decision node.