Ensemble methods have the ability to provide much needed robustness and accuracy to both supervised and unsupervised problems. Machine learning is going to evolve more and more and computations power becomes cheap and the volume of data continues to increase. In such a scenario, there is a limit to the improvement you can achieve by using a single framework and attempting to improve its predictive power (using modification in variables).
Ensemble Modeling follows the philosophy of ‘Unity in Strength’ i.e. combination of diversified base models strengthens weak models. The success of ensemble techniques spreads across multiple disciplines like recommendation systems, anomaly detection, stream mining, and web applications where the need for combination of competing models is ubiquitous.
If you wish to experience the powerful nature of ensemble, try using supervised and unsupervised model for a single task and merge their results. You’ll notice that the merger delivered better performance.
Ensemble of diverse and high performance models are better than individual models.
Forward Selection of learners : Imagine a scenario where we have 1000 learner outputs. We start with an empty bag and then in every iteration we add a new learner which benefits the bag on performance metric.
Selection with Replacement : To select a new addition for the bag, we put our hand in the stack of 1000 leaner and pull out the best of the lot. Even if a learner is found to be a good fit, we’ll still use this learner in the next iteration stack.
Bagging of Ensemble Models : Ensemble learners are prone to over-fitting. To avoid this, we take a sample to try ensembling. Once we are done, we again use another sample. Finally, we bag all these models together using simple average of predictions or maximum votes.
Even though bagging tackles majority of over-fitting cases, still it is good to be cautious about over-fitting in ensemble learners. A possible solution is set aside one set of population untouched and try performance metrics using this untouched test population. The two methods mentioned are no way exhaustive list of possible ensemble techniques. Ensemble is more of an art than science. Most of the master kagglers are masters of this art.
Did you find this article useful? Have you tried anything else to find optimal weights? I’ll be happy to hear from you in the comments section below.
i’m a diploma holder in electronics engineering and also have 2.5+years work experience in mobile service industry. I have completed my BCA this year in distance mode.
i want to be a business analytics in multiple domain.
I want to persue MBA,but i’m confused that should i go to regular mode or from distance mode and also the specilization.
Please put such generic posts on the forum. That ways you will also get more opinions on your questions for industry leaders. Also that will help us keep this arena of article zone, specific enough.
The post is not specific to any particular dataset. In case you need to experiment this code, you can download a few datasets from Kaggle.
I don’t understand the step 4? What is its need? What models are being talked about when we have not created any model as of yet? And what is it doing with columns of the train set to determine rmsle?