title: XGboost - increasing training error "mae"
body: <p>I'm running XGBoost XGBRegressor with Python and dataset that looks like this: </p>
  
  <p><strong><code>click_id | manufacturer | category    | delivery_time | price | revenue</code></strong> <br/>
  <code>1        |10            | 100        | 24             | 100   | 0</code><br/>
  <code>2        |10            | 100        | 24             | 100   | 0</code><br/>
  <code>3        |10            | 100        | 24             | 100   | 0</code><br/>
  <code>4        |10            | 100        | 24             | 100   | 120</code><br/>
  <code>5        |20            | 200        | 48             | 200   | 0</code><br/></p>
  
  <p>Revenue is dependent variable and the rest of variables are features. </p>
  
  <p>When I run XGBRegressor and set <em>eval_metric</em> as "<em>mae</em>" (mean absolute error) the training and validation errors are constantly increasing. How can training error increase? Is there any case (any combination of model parameters or weird data points) that might cause xgboost training error to increase? </p>
  
  <p>This is the code:</p>
  
  <pre><code>model = XGBRegressor(
      learning_rate=0.1,
      n_estimators=200,
      max_depth=5,
      min_child_weight=1,
      gamma=0,
      subsample=0.9,
      colsample_bytree=0.9,
      reg_alpha=10,
      nthread=4)
  
  model.fit(X_train, y_train, 
            eval_set=[(X_train, y_train), (X_test, y_test)], eval_metric='mae')
  </code></pre>
  
  <p>When <em>eval_metric</em> is set as "<em>rmse</em>" training error is decreasing as expected.</p>