Title: Order-Sensitivity and Equivariance of Scoring Functions,
Abstract: The relative performance of competing point forecasts is usually measured in
terms of loss or scoring functions. It is widely accepted that these scoring
function should be strictly consistent in the sense that the expected score is
minimized by the correctly specified forecast for a certain statistical
functional such as the mean, median, or a certain risk measure. Thus, strict
consistency opens the way to meaningful forecast comparison, but is also
important in regression and M-estimation. Usually strictly consistent scoring
functions for an elicitable functional are not unique. To give guidance on the
choice of a scoring function, this paper introduces two additional quality
criteria. Order-sensitivity opens the possibility to compare two deliberately
misspecified forecasts given that the forecasts are ordered in a certain sense.
On the other hand, equivariant scoring functions obey similar equivariance
properties as the functional at hand - such as translation invariance or
positive homogeneity. In our study, we consider scoring functions for popular
functionals, putting special emphasis on vector-valued functionals, e.g. the
pair (mean, variance) or (Value at Risk, Expected Shortfall).