xgboost performance with predicted values as input I have predicted the probability of loss using different features. Now when I used this with a non-important feature to predict the probability of loss. The first one is very close. logloss was close to 0.11. However, I have few more other features, I wanted to know if the features are important or not. So, I used the new features with this predicted probability. I found volatile behavior. Not only did the performance (logloss) drop to 0.14, but the model didn't pick the predicted probability as an important feature.

My Primary Questions:


What is the reason behind this outcome?  
Should I dump every single
feature into one model, then see which feature is important ?
You could try   the next thing:,   get some   top  k predictions made with   xgboost, and use  only those as features  and   feed them  into  an LogisticRegression model.
I am currious if that will help  in case  you want  to  give a  feedback   on my proposed solution