title: Using XGboost_Regressor in Python results in very good training performance but poor in prediction
body: <p>I have been trying to use XGBregressor in python. It is by far one of the best ML techniques I have used.However, in some data sets I have very high training R-squared, but it performs really poor in prediction or testing. I have tried playing with gamma, depth, and subsampling to reduce the complexity of the model or to make sure its not overfitted but still there is a huge difference between training and testing. I was wondering if someone could help me with this: </p>
  
  <p>Below is the code I am using: </p>
  
  <pre><code>from sklearn.model_selection import train_test_split
  
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30,random_state=100)
  
  from sklearn.preprocessing import MinMaxScaler
  scaler = MinMaxScaler()
  scaler.fit(X_train)
  
  
  xgb = xgboost.XGBRegressor(colsample_bytree=0.7,
                   gamma=0,                 
                   learning_rate=0.01,
                   max_depth=1,
                   min_child_weight=1.5,
                   n_estimators=100000,                                                                    
                   reg_alpha=0.75,
                   reg_lambda=0.45,
                   subsample=0.8,
                   seed=1000) 
  </code></pre>
  
  <p>Here is the performance in training vs testing: </p>
  
  <p>Training : 
  MAE: 0.10    R^2: 0.99</p>
  
  <p>Testing: 
  MAE: 1.47      R^2: -0.89</p>