XGBoost - feature importance just depends on the location of the feature in the data I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data is by far most important and then second is second, etc.
It seems like a sign that the model is not working properly as its not really learning anything... any advice on what could be going wrong?
UPDATE:
Correlation Matrix
https://ibb.co/3shDJjD
Model Code:
params = {
  'subsample':0.5,
  'learning_rate': 0.3,
  'max_depth':8,
  'num_parallel_trees' : 20,
  'objective': 'reg:squarederror',
  'verbosity':0,
  }
watchlist = [(train, 'train'), (test, 'val')]
reg = xgb.train(params, train, num_boost_round=5, early_stopping_rounds=5, evals=watchlist)

Results:
[0] train-rmse:0.274535 val-rmse:0.27431
Multiple eval metrics have been passed: 'val-rmse' will be used for early stopping.

Will train until val-rmse hasn't improved in 5 rounds.
[1] train-rmse:0.273472 val-rmse:0.273653
[2] train-rmse:0.272796 val-rmse:0.27341
[3] train-rmse:0.272318 val-rmse:0.27334
[4] train-rmse:0.271943 val-rmse:0.273346
[5] train-rmse:0.271604 val-rmse:0.273374
[6] train-rmse:0.271218 val-rmse:0.273442
[7] train-rmse:0.270927 val-rmse:0.273529
[8] train-rmse:0.270641 val-rmse:0.273561
Stopping. Best iteration:
[3] train-rmse:0.272318 val-rmse:0.27334

Feature importance (note that 0 and 1 are first). If I change the order of the columns in the xtrain, the feature importance will also change and first two columns will always be two most important features.
https://ibb.co/QcHwbNg
You can use this chunk of code to plot the feature importance of your data. It is also possible that your data is columned with decreasing importance.
from xgboost import XGBClassifier
from xgboost import plot_importance
from matplotlib import pyplot
# load data
X = 
y = 
model = XGBClassifier()
model.fit(X, y)
plot_importance(model)
pyplot.show()