Title: Kernel Regression with Sparse Metric Learning,
Abstract: Kernel regression is a popular non-parametric fitting technique. It aims at
learning a function which estimates the targets for test inputs as precise as
possible. Generally, the function value for a test input is estimated by a
weighted average of the surrounding training examples. The weights are
typically computed by a distance-based kernel function and they strongly depend
on the distances between examples. In this paper, we first review the latest
developments of sparse metric learning and kernel regression. Then a novel
kernel regression method involving sparse metric learning, which is called
kernel regression with sparse metric learning (KR$\_$SML), is proposed. The
sparse kernel regression model is established by enforcing a mixed $(2,1)$-norm
regularization over the metric matrix. It learns a Mahalanobis distance metric
by a gradient descent procedure, which can simultaneously conduct
dimensionality reduction and lead to good prediction results. Our work is the
first to combine kernel regression with sparse metric learning. To verify the
effectiveness of the proposed method, it is evaluated on 19 data sets for
regression. Furthermore, the new method is also applied to solving practical
problems of forecasting short-term traffic flows. In the end, we compare the
proposed method with other three related kernel regression methods on all test
data sets under two criterions. Experimental results show that the proposed
method is much more competitive.