title: Grid search with LightGBM regression
body: <p>I want to train a regression model using Light GBM, and the following code works fine:</p>
  
  <pre><code>import lightgbm as lgb
  
  d_train = lgb.Dataset(X_train, label=y_train)
  params = {}
  params['learning_rate'] = 0.1
  params['boosting_type'] = 'gbdt'
  params['objective'] = 'gamma'
  params['metric'] = 'l1'
  params['sub_feature'] = 0.5
  params['num_leaves'] = 40
  params['min_data'] = 50
  params['max_depth'] = 30
  
  lgb_model = lgb.train(params, d_train, 1000)
  
  #Prediction
  y_pred=lgb_model.predict(X_test)
  mae_error = mean_absolute_error(y_test,y_pred)
  
  print(mae_error)
  </code></pre>
  
  <p>But when I proceed to using GridSearchCV, I encounter problems. I am not completely sure how to set this up correctly. I found useful sources, for example <a href="https://dataninjago.com/2017/08/04/iot-ml-tuning-hyper-parameters-using-grid-search/" rel="nofollow noreferrer">here</a>, but they seem to be working with a classifier.</p>
  
  <p>1st try:</p>
  
  <pre><code>from sklearn.metrics import make_scorer
  score_func = make_scorer(mean_absolute_error, greater_is_better=False)
  
  model = lgb.LGBMClassifier( 
      boosting_type="gbdt",
      objective='regression',
      is_unbalance=True, 
      random_state=10, 
      n_estimators=50,
      num_leaves=30, 
      max_depth=8,
      feature_fraction=0.5,  
      bagging_fraction=0.8, 
      bagging_freq=15, 
      learning_rate=0.01,    
  )
  
  params_opt = {'n_estimators':range(200, 600, 80), 'num_leaves':range(20,60,10)}
  gridSearchCV = GridSearchCV(estimator = model, 
      param_grid = params_opt, 
      scoring=score_func)
  gridSearchCV.fit(X_train,y_train)
  gridSearchCV.grid_scores_, gridSearchCV.best_params_, gridSearchCV.best_score_
  </code></pre>
  
  <p>, gives me a bunch of error before:</p>
  
  <blockquote>
    <p>"ValueError: Unknown label type: 'continuous'"</p>
  </blockquote>
  
  <p><strong>UPDATE</strong>: I made the code run switching LGBMClassifier with LGBMModel. Should I try to use LGBMRegressor too, or does this not matter? (source: <a href="https://lightgbm.readthedocs.io/en/latest/_modules/lightgbm/sklearn.html" rel="nofollow noreferrer">https://lightgbm.readthedocs.io/en/latest/_modules/lightgbm/sklearn.html</a>)</p>