Random Forest vs Gradient Boosting. Like random forests, gradient boosting is a set of decision trees. The two main differences are: ... Combining results: random forests combine results at the end of the process (by averaging or "majority rules") while gradient boosting combines results along the way.