XGBoost is it possible to prevent a feature from being used twice in the same tree? I'm using XGBoost and all its doing is using the feature in the first column of my data. My feature importance chart correlates perfectly to the position of the feature in my xtrain. If I shuffle the columns in xtrain, the feature chart changes along witht he shuffle after re-running the model.
This suggests that XGBoost is &quot;getting stuck&quot; on the first feature. Is there a way to prevent this? I was thinking by limiting how many times the same feature can appear in a tree?
If you mean &quot;how many times the same feature can appear in an [individual] tree&quot;, then
you can use max_depth to indirectly limit the number of features included in a single tree, even down to one feature. Since XGBoost is designed to use weak learners, having a lower depth value is ok.
model = XGBClassifier(max_depth=n)

However, I think the problem is not that XGBoost is getting stuck on a single feature. For example, maybe the other columns contain little or no correlation to the label. How does your model perform when you remove that column entirely. That should tell you if you only have one good feature in your dataset.