Title: Prior Information Guided Regularized Deep Learning for Cell Nucleus Detection,
Abstract: Cell nuclei detection is a challenging research topic because of limitations
in cellular image quality and diversity of nuclear morphology, i.e. varying
nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been
a topic of enduring interest with promising recent success shown by deep
learning methods. These methods train Convolutional Neural Networks (CNNs) with
a training set of input images and known, labeled nuclei locations. Many such
methods are supplemented by spatial or morphological processing. Using a set of
canonical cell nuclei shapes, prepared with the help of a domain expert, we
develop a new approach that we call Shape Priors with Convolutional Neural
Networks (SP-CNN). We further extend the network to introduce a shape prior
(SP) layer and then allowing it to become trainable (i.e. optimizable). We call
this network tunable SP-CNN (TSP-CNN). In summary, we present new network
structures that can incorporate 'expected behavior' of nucleus shapes via two
components: learnable layers that perform the nucleus detection and a fixed
processing part that guides the learning with prior information. Analytically,
we formulate two new regularization terms that are targeted at: 1) learning the
shapes, 2) reducing false positives while simultaneously encouraging detection
inside the cell nucleus boundary. Experimental results on two challenging
datasets reveal that the proposed SP-CNN and TSP-CNN can outperform
state-of-the-art alternatives.