Title: OpenML Benchmarking Suites and the OpenML100,
Abstract: We advocate the use of curated, comprehensive benchmark suites of machine
learning datasets, backed by standardized OpenML-based interfaces and
complementary software toolkits written in Python, Java and R. Major
distinguishing features of OpenML benchmark suites are (a) ease of use through
standardized data formats, APIs, and existing client libraries; (b)
machine-readable meta-information regarding the contents of the suite; and (c)
online sharing of results, enabling large scale comparisons. As a first such
suite, we propose the OpenML100, a machine learning benchmark suite of
100~classification datasets carefully curated from the thousands of datasets
available on OpenML.org.