another way of modeling ensemble based methods is boosting very popular method is xgboost which stands for extreme gradient boosting it is variant of the gradient boosting machines (gbmmodel this model is extremely popular in the data science community owing to its superior performance in several data science challenges and competitions especially on kaggle for using this modelyou can install the xgboost package in python for details on this frameworkfeel free to check out the official web site at installationmodel tuningand much more credits go to the distributed machine learning communitypopularly known as dmlc for creating the xgboost framework along with the popular mxnet deep learning framework gradient boosting using the principles of boosting methodology for ensemblingwhich we discussed earlierand it uses gradient descent to minimize error or loss when adding new weak base learners going into details of the model internals would be out of the current scopebut we recommend checking out trees and the principle of xgboost we trained basic xgboost model first on our data and obtained an overall accuracy of around after tuning the model with grid searchwe trained the model with the following parameter values and evaluated its performance on the test data (detailed step-by-step snippets are available in the jupyter notebookin [ ]import os mingw_path ' :\mingw- \mingw \binos environ['path'mingw_path ';os environ['path'import xgboost as xgb train the model on tuned hyperparameters wqp_xgb_model xgb xgbclassifier(seed= max_depth= learning_rate= n_estimators= wqp_xgb_model fit(wqp_train_sxwqp_train_yevaluate and predict performance wqp_xgb_predictions wqp_xgb_model predict(wqp_test_sxmeu display_model_performance_metrics(true_labels=wqp_test_ypredicted_labels=wqp_xgb_predictionsclasses=wqp_label_namesfigure - model performance metrics for tuned xgboost model for wine quality predictive model the model prediction results on the test dataset depict an overall score and model accuracy of approximately %as seen in figure - though random forests perform slightly betterit definitely performs better than basic model like decision tree try adding more hyperparameters to tune the model and see if you can get better model if you do find onefeel free to send pull request to our repositorywe have successfully built decent wine quality classifier using several techniques and also seen the importance of model tuning and validation let' take our best model and run some model interpretation tasks on it to try to understand it better to start withwe can look at the feature importance ranks given to the various features in the dataset the following snippet shows comparative feature importance plots using skater as well as the default feature importances obtained from the scikit-learn model itself we build an interpretation and inmemorymodel object using skaterwhich will be useful for our future analyses in model interpretation