Title: Convolutional Analysis Operator Learning: Acceleration, Convergence, Application, and Neural Networks,
Abstract: Convolutional operator learning is increasingly gaining attention in many
signal processing and computer vision applications. Learning kernels has mostly
relied on so-called local approaches that extract and store many overlapping
patches across training signals. Due to memory demands, local approaches have
limitations when learning kernels from large datasets -- particularly with
multi-layered structures, e.g., convolutional neural network (CNN) -- and/or
applying the learned kernels to high-dimensional signal recovery problems. The
so-called global approach has been studied within the "synthesis" signal model,
e.g., convolutional dictionary learning, overcoming the memory problems by
careful algorithmic designs. This paper proposes a new convolutional analysis
operator learning (CAOL) framework in the global approach, and develops a new
convergent Block Proximal Gradient method using a Majorizer (BPG-M) to solve
the corresponding block multi-nonconvex problems. To learn diverse filters
within the CAOL framework, this paper introduces an orthogonality constraint
that enforces a tight-frame (TF) filter condition, and a regularizer that
promotes diversity between filters. Numerical experiments show that, for tight
majorizers, BPG-M significantly accelerates the CAOL convergence rate compared
to the state-of-the-art method, BPG. Numerical experiments for sparse-view
computational tomography show that CAOL using TF filters significantly improves
reconstruction quality compared to a conventional edge-preserving regularizer.
Finally, this paper shows that CAOL can be useful to mathematically model a
CNN, and the corresponding updates obtained via BPG-M coincide with core
modules of the CNN.