Title: Automatic Gradient Boosting,
Abstract: Automatic machine learning performs predictive modeling with high performing
machine learning tools without human interference. This is achieved by making
machine learning applications parameter-free, i.e. only a dataset is provided
while the complete model selection and model building process is handled
internally through (often meta) optimization. Projects like Auto-WEKA and
auto-sklearn aim to solve the Combined Algorithm Selection and Hyperparameter
optimization (CASH) problem resulting in huge configuration spaces. However,
for most real-world applications, the optimization over only a few different
key learning algorithms can not only be sufficient, but also potentially
beneficial. The latter becomes apparent when one considers that models have to
be validated, explained, deployed and maintained. Here, less complex model are
often preferred, for validation or efficiency reasons, or even a strict
requirement. Automatic gradient boosting simplifies this idea one step further,
using only gradient boosting as a single learning algorithm in combination with
model-based hyperparameter tuning, threshold optimization and encoding of
categorical features. We introduce this general framework as well as a concrete
implementation called autoxgboost. It is compared to current AutoML projects on
16 datasets and despite its simplicity is able to achieve comparable results on
about half of the datasets as well as performing best on two.