title: How to build voting classifier in sklearn when the individual classifiers are being fit with different datasets?
body: <p>I'm building a classifier using highly unbalanced data. The strategy I'm interesting in testing is ensembling a model using <strong>3 different resampled datasets</strong>. In other words, each dataset will have <em>all</em> the samples from the rare class, but <em>only</em> n samples of the abundant class (<a href="https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html" rel="noreferrer">technique #4 mentioned in this article</a>). </p>
  
  <p>I want to fit 3 different <code>VotingClassifiers</code> on each resampled dataset, and then combine the results of the individual models using <em>another</em> <code>VotingClassifier</code> (or similar). I know that building a single voting classifier looks like this: </p>
  
  <pre><code># First Model
  rnd_clf_1 = RandomForestClassifier()
  xgb_clf_1 = XGBClassifier()
  
  voting_clf_1 = VotingClassifier(
      estimators = [
          ('rf', rnd_clf_1), 
          ('xgb', xgb_clf_1),
      ],
      voting='soft'
  )
  
  # And I can fit it with the first dataset this way:
  voting_clf_1.fit(X_train_1, y_train_1)
  </code></pre>
  
  <p>But how to stack the three of them if they are fitted on different datasets? For example, if I had three fitted models (see code below), I could build a function that calls the <code>.predict_proba()</code> method on each of the models and then "manually" averages the individual probabilities. </p>
  
  <p><strong>But... is there a better way?</strong></p>
  
  <pre><code># Fitting the individual models... but how to combine the predictions?
  voting_clf_1.fit(X_train_1, y_train_1)
  voting_clf_2.fit(X_train_2, y_train_2)
  voting_clf_3.fit(X_train_3, y_train_3)
  </code></pre>
  
  <p>Thanks! </p>