title: Feature importance calculation for gradient boosted regression tree versus random forest
body: <p>On data with a few features I train a random forest for regression purposes and also gradient boosted regression trees. For both I calculate the feature importance, I see that these are rather different, although they achieve similar scores.</p>
  
  <p>For the random forest regression:</p>
  
  <pre><code>MAE: 59.11
  RMSE: 89.11
  
  Importance:
  
  Feature 1: 64.87
  Feature 2: 0.10
  Feature 3: 29.03
  Feature 4: 0.09
  Feature 5: 5.89
  </code></pre>
  
  <p>For the gradient boosted regression trees:</p>
  
  <pre><code>MAE: 58.70
  RMSE: 90.59
  
  Feature 1: 65.18
  Feature 2: 5.67
  Feature 3: 13.61
  Feature 4: 4.26
  Feature 5: 11.27
  </code></pre>
  
  <p>Why is this ? I thought maybe because with gradient boosted regression trees, the trees are more shallow than with random forests. But I am not sure.</p>