title: Very weird behaviour of xgboost in R
body: <p><br />
  I am learning to use the XGBoost package in R and I encountered some very weird behaviour that I'm not sure how to explain. Perhaps someone can give me some directions. I simplified the R code as much as possible:</p>
  <pre><code>rm(list = ls())
  library(xgboost)
  setwd(&quot;/home/my_username/Documents/R_files&quot;)
  
  my_data &lt;- read.csv(&quot;my_data.csv&quot;)
  my_data$outcome_01 = ifelse(my_data$outcome_continuous &gt; 0.0, 1, 0)
  
  reg_features = c(&quot;feature_1&quot;, &quot;feature_2&quot;)
  class_features = c(&quot;feature_1&quot;, &quot;feature_3&quot;)
  
  set.seed(93571)
  train_data = my_data[seq(1, nrow(my_data), 2), ]
  
  mm_reg_train = model.matrix(~ . + 0, data = train_data[, reg_features])
  train_DM_reg = xgb.DMatrix(data = mm_reg_train, label = train_data$outcome_continuous)
  
  var_nrounds = 190
  xgb_reg_model = xgb.train(data = train_DM_reg, booster = &quot;gbtree&quot;, objective = &quot;reg:squarederror&quot;,
                            nrounds = var_nrounds, eta = 0.07,
                            max_depth = 5, min_child_weight = 0.8, subsample = 0.6, colsample_bytree = 1.0,
                            verbose = F)
  
  mm_class_train = model.matrix(~ . + 0, data = train_data[, class_features])
  train_DM_class = xgb.DMatrix(data = mm_class_train, label = train_data$outcome_01)
  
  xgb_class_model = xgb.train(data = train_DM_class, booster = &quot;gbtree&quot;, objective = &quot;binary:logistic&quot;,
                              eval_metric = 'auc', nrounds = 70, eta = 0.1,
                              max_depth = 3, min_child_weight = 0.5, subsample = 0.75, colsample_bytree = 0.5,
                              verbose = F)
  
  probabilities = predict(xgb_class_model, newdata = train_DM_class, type = &quot;response&quot;)
  print(paste0(&quot;simple check: &quot;, sum(probabilities)), quote = F)
  </code></pre>
  <p>Here is the problem: The outcome of <code>sum(probabilities)</code> depends on the value of <code>var_nrounds</code>!</p>
  <p>How could that be? After all <code>var_nrounds</code> enters only in <code>xgb_reg_model</code>, while the probabilities are computed with the <code>xgb_class_model</code> that does (should) not know anything about the value of <code>var_nrounds</code>. The only thing that I change in this code is the value of <code>var_nrounds</code> and yet the sum of probabilities changes when I rerun it. It also changes deterministically, i.e., with <code>var_nrounds = 190</code> I always get (with my data) <code>5324.3</code> and with <code>var_nrounds = 285</code>:<code>5322.8</code>. However, if I remove the line <code>set.seed(93571)</code>, then the result changes non-deterministically every time I rerun the code.</p>
  <p>Could it be that XGBoost has some sort of in-built stochastic behaviour that changes depending on the number of rounds run beforehand in another model and that also gets controlled by setting a seed somewhere in the code before training the XGBoost? Any ideas?</p>