title: How can I use R^2 as an evaluation metric when modeling?
body: <p>I am using Python to train an XGBoost Regressor on a 25 feature column dataset and SKlearn's GridSearchCV for parameter tuning. GridSearchCV allows you to choose your scorer with the 'scoring' parameter, and <code>r2</code> is a valid option. </p>
  
  <pre><code>grid = GridSearchCV(mdl, param_grid=params, verbose=1, cv=kfold,
                  n_jobs=-1, error_score='raise',scoring='r2')`
  </code></pre>
  
  <p>However, when I look to use <code>r2</code> as my 'eval_metric' in the <code>grid.fit()</code> function, I don't have a great way to use <code>r2</code>. </p>
  
  <pre><code>grid.fit(X_train, y_train, eval_set=[(X_test, y_test)],
           eval_metric='rmse', early_stopping_rounds=150)
  </code></pre>
  
  <p>I have tried using sklearns built-in <code>r2_score</code> method, but there are a few issues. The first being, an <code>r2 score</code> is calculated given the <code>y_test</code> set against the <code>y_pred</code> set. And in order to have a <code>y_pred</code> set, we need to fit the model. So you can see I'm running into a looping issue.</p>
  
  <p>I have tried a few things to get around this. The first being training the model and making predictions inside the eval_metric variable like below:</p>
  
  <pre><code>grid.fit(X_train, y_train, eval_set=[(X_test, y_test)],
           eval_metric=r2_score(y_test, mdl.predict(X_test)), early_stopping_rounds=150)
  </code></pre>
  
  <p>But I am given the following error:
  <code>xgboost.core.XGBoostError: need to call fit beforehand</code></p>
  
  <p>Which makes sense. </p>
  
  <p>Is there some way that I can grab the current parameters that the GridSearchCV is using, create and store predictions, and then use the <code>r2_score</code> as the eval_metric?</p>
  
  <p>My thoughts are this. The r2 score is a standard evaluation metric on a scale of 0 to 1 (1 being a perfect fit). This is a metric that if there were a way to standardize optimizing it, would have a very far reach across almost all machine learning. </p>