Title: Structure-aware error bounds for linear classification with the zero-one loss,
Abstract: We prove risk bounds for binary classification in high-dimensional settings
when the sample size is allowed to be smaller than the dimensionality of the
training set observations. In particular, we prove upper bounds for both
'compressive learning' by empirical risk minimization (ERM) (that is when the
ERM classifier is learned from data that have been projected from
high-dimensions onto a randomly selected low-dimensional subspace) as well as
uniform upper bounds in the full high-dimensional space. A novel tool we employ
in both settings is the 'flipping probability' of Durrant and Kaban (ICML 2013)
which we use to capture benign geometric structures that make a classification
problem 'easy' in the sense of demanding a relatively low sample size for
guarantees of good generalization. Furthermore our bounds also enable us to
explain or draw connections between several existing successful classification
algorithms. Finally we show empirically that our bounds are informative enough
in practice to serve as the objective function for learning a classifier (by
using them to do so).