DSTARS: A multi-target deep structure for tracking asynchronous regressor stacking

Several applications of supervised learning involve the prediction of multiple continuous target variables from a dataset. When the target variables exhibit statistical dependencies among them, a multi-target regression (MTR) modelling permits to improve the predictive performance in comparison to induce a separate model for each target. Apart from describing the dependencies among the targets, the MTR methods could offer better performance and less overfitting than traditional single-target (ST) methods. A group of MTR methods have addressed this demand, but there are still many possibilities for further improvements. This paper presents a novel MTR method called Deep Structure for Tracking Asynchronous Regressor Stacking (DSTARS), which overcomes some existing gaps in the current solutions. DSTARS extends the Stacked Single-Target (SST) approach by combining multiple stacked regressors into a deep structure. In this sense, it is able to boost the predictive performance by successively improving the predictions for the targets. Besides, DSTARS exploits the dependency of each target individually by tracking an asynchronous number of stacked regressors. Additionally, our proposal explores the inter-targets dependencies by exposing and measuring them through a nonlinear metric of variable importance. We compared DSTARS to SST, Ensemble of Regressor Chains (ERC) and Multi-objective Random Forest (MORF). Also, the ST strategy with different algorithms was used to compute independent regressions for each target. We used Random Forest (RF) and Support Vector Machine (SVM) as base-learners to investigate the prediction capability of algorithms belonging to different machine learning paradigms. The experiments carried out on eighteen diverse datasets showed that the proposed method was significantly better than the other compared approaches.