title: SKLearn Stacking Classifier giving bad results
body: <p>I am making a hybrid ensemble model with scikit learn. I tried to use stacking classfier to ensemble models where I used random forest, SVM and naive bayes with Logistic regression as meta estimator. I am using the heart disease dataset on <a href="https://www.kaggle.com/ronitf/heart-disease-uci" rel="nofollow noreferrer">kaggle</a> . The problem is that the solo techniques are performing better than the hybrid results. My hybrid model is using techniques with gridsearchCV of 5 fold validation and other best parameters. Whilst my solo techniques to which I am comparing model to , have default parameters. What I am doing wrong.</p>
  <p>Logistic Regression :</p>
  <pre><code>C = [0.01,0.1, 1, 5, 10]
  
  Log = LogisticRegression()
  
  parameters = {'C': [.1 ,2, 5, 10, 15, 20]}
  
  log_regressor = GridSearchCV(Log, parameters,cv =5)
  log_regressor.fit(X_train, y_train)
  log_regressor.best_params_
  log_regressor.fit(X_train,y_train)
  accuracy89 = log_regressor.score(X_test,y_test)
  
  print('Logistic Regression Accuracy --&gt;',((accuracy89)*100))
  </code></pre>
  <p>Naive bayes:</p>
  <pre><code>params_NB = {'var_smoothing': np.logspace(0,-9, num=100)}
  NB = GaussianNB()
  nb_regressor = GridSearchCV(NB, params_NB,n_jobs = -1 ,cv =5)
  nb_regressor.fit(X_train,y_train)
  accuracy76 = nb_regressor.score(X_test,y_test)
  print('Naive Bayes Accuracy --&gt;',((accuracy76)*100))
  nb_regressor.best_params_
  </code></pre>
  <p>Similarly , SVC and Random forest were trained. Then the best parameter model was used here in stacking classifier:</p>
  <pre><code>estimators = [
      ('knn',model3_grid),
      ('svc',svm_regressor),
      ('nb',nb_regressor),
      ('rf',rf_classifier),
  ]
  
  stackingCLF = StackingClassifier(estimators = estimators, verbose = 2 ,final_estimator = log_regressor, cv=5)
  stackingCLF.fit(X_train, y_train)
  StackedScore = stackingCLF.score(X_test, y_test)*100
  print(StackedScore)
  </code></pre>
  <p>It is giving me 85 % accuracy but the solo tech. results are:</p>
  <pre><code>'DT': 0.7166666666666667,
   'KNN': 0.7833333333333333,
   'Logistic Regression': 0.8666666666666667,
   'NB': 0.5,
   'RF': 0.8166666666666667,
   'SVC': 0.8
  </code></pre>