One of the current challenges in machine learning is how to deal with data coming at increasing rates in data streams. New predictive learning strategies are needed to cope with the high throughput data and concept drift. One of the data stream mining tasks where new learning strategies are needed is multi-target regression, due to its applicability in a high number of real world problems. While reliable and effective learning strategies have been proposed for batch multi-target regression, few have been proposed for multi-target online learning in data streams. Besides, the existing solutions do not consider the occurrence of inter-target correlations when making predictions. In this work, we propose an online learning strategy for multi-target regression in data streams. The proposed strategy extends a single target decision tree online learning algorithm to allow multi-target online regression learning in data streams. For such, the proposed strategy, called Stacked Single-target Hoeffding Tree (SST-HT), uses the inter-target dependencies as an additional information source to enhance predictive accuracy. Throughout an extensive experimental setup, we evaluate our proposal against state-of-the-art decision tree-based algorithms for online multi-target regression. According to the experimental results, SST-HT presents superior predictive accuracy, with a small increase in the processing time and memory requirements.
@cite_8: As in batch learning, one may identify a class of streaming real-world problems which require the modeling of several targets simultaneously. Due to the dependencies among the targets, simultaneous modeling can be more successful and informative than creating independent models for each target. As a result one may obtain a smaller model able to simultaneously explain the relations between the input attributes and the targets. This problem has not been addressed previously in the streaming setting. We propose an algorithm for inducing multi-target model trees with low computational complexity, based on the principles of predictive clustering trees and probability bounds for supporting splitting decisions. Linear models are computed for each target separately, by incremental training of perceptrons in the leaves of the tree. Experiments are performed on synthetic and real-world datasets. The multi-target regression tree algorithm produces equally accurate and smaller models for simultaneous prediction of all the target attributes, as compared to a set of independent regression trees built separately for each target attribute. When the regression surface is smooth, the linear models computed in the leaves significantly improve the accuracy for all of the targets.
Following the trend of using tree-based methods for streaming scenarios, proposed an extension for the FIMT-MT algorithm called iSOUP-Tree (incremental Structured Output Prediction Tree). Their proposal builds upon the research of @cite_8 by enabling the support to categorical features and employing an adaptive prediction model at the leaves. Instead of only using perceptrons at the leaves, iSOUP-Tree also maintains a mean predictor for each target and selects the best current model by monitoring a faded error metric for each of them. The authors adapted the iSOUP-Tree algorithm for multiple settings, including ensembles (Bagging and Random Forest) and Option Trees , and employed all the variations of the MTR method to tackle multi-label classification tasks .