title: XGBRegressor.predict() returns error after pickle.load()
body: <p>I have trained XGBRegressor model using sklearn interface. Relevant code is as follows:</p>
  
  <pre><code>def xgb_regressor_wrapper(X_train, y_train):
      xgb_regressor = XGBRegressor(objective='reg:linear', n_estimators=1000, learning_rate=0.01, base_score=0.005)
      xgb_regressor.fit(X=X_train, y=y_train) #, eval_set=[(X_test, y_test)], verbose=True)
      return xgb_regressor
  
  def save_regressor(station, feature, regressor):
      fname = generate_regressor_fname(station, feature)
      pickle.dump(regressor, open(fname, "wb" ))
  
  # regressor_list dict contains wrapper functions
  # I currently have XGBRegressor and CatBoostRegressor in the list.
  regressor_wrapper = regressor_list.get(name) 
  
  # Create and fit XGBRegressor
  regressor = regressor_wrapper(X_train, y_train)
  
  # Save regressor
  save_regressor(station_id, feature, best_regressor)
  </code></pre>
  
  <p>Some time later, I use the following code to reload the regressor, and do the predictions:</p>
  
  <pre><code>def load_regressor(station, feature):
      fname = generate_regressor_fname(station, feature)
      return pickle.load(open(fname, "rb" ))
  
  # Load the regressor
  regressor = load_regressor(station_id, feature)
  
  # Do the prediction
  y_predict = regressor.predict(X_test)
  </code></pre>
  
  <p>I get the following error:</p>
  
  <pre><code>  File "regressor_stuff.py", line 169, in regressor_check_for_station_feature
      y_predict = regressor.predict(X_test)
    File "D:\Anaconda\envs\Deep\lib\site-packages\xgboost\sklearn.py", line 268, in predict
      return self.booster().predict(test_dmatrix,
  TypeError: 'str' object is not callable
  </code></pre>
  
  <p>After some debugging, I saw <code>self.booster</code> actually stores string 'gbtree'. After training regressor for hundereds of features (which took some days by the way) this was not cool.</p>
  
  <p>Any suggestion for why this could have been happened?</p>
  
  <p>My current workaround is to reconstruct the XGBBooster as follows:</p>
  
  <pre><code># Load the regressor
  if isinstance(regressor, XGBRegressor):
      regressor = XGBRegressor()
      r = pickle.load(open(fname, "rb" ))
      print r.get_xgb_params()
      regressor._Booster = r._Booster
      regressor.set_params(**r.get_xgb_params())
  
  # Do the prediction
  y_predict = regressor.predict(X_test)
  </code></pre>
  
  <p>Thank you</p>
  
  <p>Kursat</p>