model of similar complexity in contrast to random forestswhere higher n_esti mators value is always betterincreasing n_estimators in gradient boosting leads to more complex modelwhich may lead to overfitting common practice is to fit n_estimators depending on the time and memory budgetand then search over different learning_rates another important parameter is max_depth (or alternatively max_leaf_nodes)to reduce the complexity of each tree usually max_depth is set very low for gradient boosted modelsoften not deeper than five splits kernelized support vector machines the next type of supervised model we will discuss is kernelized support vector machines we explored the use of linear support vector machines for classification in "linear models for classificationon page kernelized support vector machines (often just referred to as svmsare an extension that allows for more complex models that are not defined simply by hyperplanes in the input space while there are support vector machines for classification and regressionwe will restrict ourselves to the classification caseas implemented in svc similar concepts apply to support vector regressionas implemented in svr the math behind kernelized support vector machines is bit involvedand is beyond the scope of this book you can find the details in of hastietibshiraniand friedman' the elements of statistical learning howeverwe will try to give you some sense of the idea behind the method linear models and nonlinear features as you saw in figure - linear models can be quite limiting in low-dimensional spacesas lines and hyperplanes have limited flexibility one way to make linear model more flexible is by adding more features--for exampleby adding interactions or polynomials of the input features let' look at the synthetic dataset we used in "feature importance in treeson page (see figure - )in[ ]xy make_blobs(centers= random_state= mglearn discrete_scatter( [: ] [: ]yplt xlabel("feature "plt ylabel("feature " supervised learning