title: How to find the best degree of polynomials?
body: <p>I'm new to Machine Learning and currently got stuck with this.
  First I use linear regression to fit the training set but get very large RMSE. Then I tried using polynomial regression to reduce the bias.</p>
  
  <pre><code>import numpy as np
  from sklearn.linear_model import LinearRegression
  from sklearn.preprocessing import PolynomialFeatures
  from sklearn.metrics import mean_squared_error
  
  poly_features = PolynomialFeatures(degree=2, include_bias=False)
  X_poly = poly_features.fit_transform(X)
  poly_reg = LinearRegression()
  poly_reg.fit(X_poly, y)
  
  poly_predict = poly_reg.predict(X_poly)
  poly_mse = mean_squared_error(X, poly_predict)
  poly_rmse = np.sqrt(poly_mse)
  poly_rmse
  </code></pre>
  
  <p>Then I got slightly better result than linear regression, then I continued to set degree = 3/4/5, the result kept getting better. But it might be somewhat overfitting as degree increased.</p>
  
  <p>The best degree of polynomial should be the degree that generates the lowest RMSE in cross validation set. But I don't have any idea how to achieve that. Should I use GridSearchCV? or any other method?</p>
  
  <p>Much appreciate if you could me with this.</p>