Title: Big Data Classification Using Augmented Decision Trees,
Abstract: We present an algorithm for classification tasks on big data. Experiments
conducted as part of this study indicate that the algorithm can be as accurate
as ensemble methods such as random forests or gradient boosted trees. Unlike
ensemble methods, the models produced by the algorithm can be easily
interpreted. The algorithm is based on a divide and conquer strategy and
consists of two steps. The first step consists of using a decision tree to
segment the large dataset. By construction, decision trees attempt to create
homogeneous class distributions in their leaf nodes. However, non-homogeneous
leaf nodes are usually produced. The second step of the algorithm consists of
using a suitable classifier to determine the class labels for the
non-homogeneous leaf nodes. The decision tree segment provides a coarse segment
profile while the leaf level classifier can provide information about the
attributes that affect the label within a segment.