Title: Deep Hyperspherical Learning,
Abstract: Convolution as inner product has been the founding basis of convolutional
neural networks (CNNs) and the key to end-to-end visual representation
learning. Benefiting from deeper architectures, recent CNNs have demonstrated
increasingly strong representation abilities. Despite such improvement, the
increased depth and larger parameter space have also led to challenges in
properly training a network. In light of such challenges, we propose
hyperspherical convolution (SphereConv), a novel learning framework that gives
angular representations on hyperspheres. We introduce SphereNet, deep
hyperspherical convolution networks that are distinct from conventional inner
product based convolutional networks. In particular, SphereNet adopts
SphereConv as its basic convolution operator and is supervised by generalized
angular softmax loss - a natural loss formulation under SphereConv. We show
that SphereNet can effectively encode discriminative representation and
alleviate training difficulty, leading to easier optimization, faster
convergence and comparable (even better) classification accuracy over
convolutional counterparts. We also provide some theoretical insights for the
advantages of learning on hyperspheres. In addition, we introduce the learnable
SphereConv, i.e., a natural improvement over prefixed SphereConv, and
SphereNorm, i.e., hyperspherical learning as a normalization method.
Experiments have verified our conclusions.