Title: Deep Networks with Shape Priors for Nucleus Detection,
Abstract: Detection of cell nuclei in microscopic images is a challenging research
topic, because of limitations in cellular image quality and diversity of
nuclear morphology, i.e. varying nuclei shapes, sizes, and overlaps between
multiple cell nuclei. This has been a topic of enduring interest with promising
recent success shown by deep learning methods. These methods train for example
convolutional neural networks (CNNs) with a training set of input images and
known, labeled nuclei locations. Many of these methods are supplemented by
spatial or morphological processing. We develop a new approach that we call
Shape Priors with Convolutional Neural Networks (SP-CNN) to perform
significantly enhanced nuclei detection. A set of canonical shapes is prepared
with the help of a domain expert. Subsequently, we present a new network
structure that can incorporate `expected behavior' of nucleus shapes via two
components: {\em learnable} layers that perform the nucleus detection and a
{\em fixed} processing part that guides the learning with prior information.
Analytically, we formulate a new regularization term that is targeted at
penalizing false positives while simultaneously encouraging detection inside
cell nucleus boundary. Experimental results on a challenging dataset reveal
that SP-CNN is competitive with or outperforms several state-of-the-art
methods.