moving forward with our mission of improving our wine quality predictive modellet' look at some ensemble modeling methods ensemble models are typically machine learning models that combine or take weighted (average\majorityvote of the predictions of each of the individual base model estimators that have been built using supervised methods of their own the ensemble is expected to generalize better over underlying databe more robustand make superior predictions as compared to each individual base model ensemble models can be categorized under three major families bagging methodsthe term bagging stands for bootstrap aggregatingwhere the ensemble model tries to improve prediction accuracy by combining predictions of individual base models trained over randomly generated training samples bootstrap samplesi independent samples with replacementare taken from the original training dataset and several base models are built on these sampled datasets at any instancean average of all predictions from the individual estimators is taken for the ensemble model to make its final prediction random sampling tries to reduce model variancereduce overfittingand boost prediction accuracy examples include the very popular random forests boosting methodsin contrast to bagging methodswhich operate on the principle of combining or averagingin boosting methodswe build the ensemble model incrementally by training each base model estimator sequentially training each model involves putting special emphasis on learning the instances which it previously misclassified the idea is to combine several weak base learners to form powerful ensemble weak learners are trained sequentially over multiple iterations of the training data with weight modifications inserted at each retrain phase at each re-training of weak base learnerhigher weights are assigned to those training instances which were misclassified previously thusthese methods try to focus on training instances which it wrongly predicted in the previous training sequence boosted models are prone to over-fitting so one should be very careful examples include gradient boostingadaboostand the very popular xgboost stacking methodsin stacking based methodswe first build multiple base models over the training data then the final ensemble model is built by taking the output predictions from these models as its additional inputs for training to make the final prediction let' now try building model using random forestsa very popular bagging method in the random forest modeleach base learner is decision tree model trained on bootstrap sample of the training data besides thiswhen we want to split decision node in the treethe split is chosen from random subset of all the features instead of taking the best split from all the features due to the introduction of this randomnessbias increases and when we average the result from all the trees in the forestthe overall variance decreasesgiving us robust ensemble model which generalizes well we will be using the randomforestclassifier from scikit-learnwhich averages the probabilistic prediction from all the trees in the forest for the final prediction instead of taking the actual prediction votes and then averaging it in [ ]from sklearn ensemble import randomforestclassifier train the model wqp_rf randomforestclassifier(wqp_rf fit(wqp_train_sxwqp_train_ypredict and evaluate performance wqp_rf_predictions wqp_rf predict(wqp_test_sxmeu display_model_performance_metrics(true_labels=wqp_test_ypredicted_labels=wqp_rf_predictionsclasses=wqp_label_names