title: Using GridSearchCV with AdaBoost and DecisionTreeClassifier
body: <p>I am attempting to tune an AdaBoost Classifier ("ABT") using a DecisionTreeClassifier ("DTC") as the base_estimator.   I would like to tune <strong>both</strong> ABT and DTC parameters simultaneously, but am not sure how to accomplish this - pipeline shouldn't work, as I am not "piping" the output of DTC to ABT.     The idea would be to iterate hyper parameters for ABT and DTC in the GridSearchCV estimator. </p>
  
  <p>How can I specify the tuning parameters correctly? </p>
  
  <p>I tried the following, which generated an error below. </p>
  
  <pre><code>[IN]
  from sklearn.tree import DecisionTreeClassifier
  from sklearn.ensemble import AdaBoostClassifier
  from sklearn.grid_search import GridSearchCV
  
  param_grid = {dtc__criterion : ["gini", "entropy"],
                dtc__splitter :   ["best", "random"],
                abc__n_estimators: [none, 1, 2]
               }
  
  
  DTC = DecisionTreeClassifier(random_state = 11, max_features = "auto", class_weight = "auto",max_depth = None)
  
  ABC = AdaBoostClassifier(base_estimator = DTC)
  
  # run grid search
  grid_search_ABC = GridSearchCV(ABC, param_grid=param_grid, scoring = 'roc_auc')
  
  [OUT]
  ValueError: Invalid parameter dtc for estimator AdaBoostClassifier(algorithm='SAMME.R',
        base_estimator=DecisionTreeClassifier(class_weight='auto', criterion='gini', max_depth=None,
          max_features='auto', max_leaf_nodes=None, min_samples_leaf=1,
          min_samples_split=2, min_weight_fraction_leaf=0.0,
          random_state=11, splitter='best'),
        learning_rate=1.0, n_estimators=50, random_state=11)
  </code></pre>