Unsupervised learning can leverage large-scale data sources without the need for annotations. In this context, deep learning-based auto encoders have shown great potential in detecting anomalies in medical images. However, state-of-the-art anomaly scores are still based on the reconstruction error, which lacks in two essential parts: it ignores the model-internal representation employed for reconstruction, and it lacks formal assertions and comparability between samples. We address these shortcomings by proposing the Context-encoding Variational Autoencoder (ceVAE) which combines reconstruction- with density-based anomaly scoring. This improves the sample- as well as pixel-wise results. In our experiments on the BraTS-2017 and ISLES-2015 segmentation benchmarks, the ceVAE achieves unsupervised ROC-AUCs of 0.95 and 0.89, respectively, thus outperforming state-of-the-art methods by a considerable margin.
@cite_5: 
@cite_25: We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.
@cite_8: In the development of large-scale knowledge networks much recent progress has been inspired by connections to neurobiology. An important component of any "neural" network is an accompanying learning algorithm. Such an algorithm, to be biologically plausible, must work for very large numbers of units. Studies of large-scale systems have so far been restricted to systems Without internal units (units With no direct connections to the input or output). Internal units are crucial to such systems as they are the means by which a system can encode high-order regularities (or invariants) that are Implicit in its inputs and outputs. Computer simulations of learning using internal units have been restricted to small-scale systems. This paper describes away of coupling autoassociative learning modules Into hierarchies that should greatly improve the performance of learning algorithms in large-scale systems. The Idea has been tested experimentally with positive results.
An Autoencoder (AE) is trained to reconstruct its input @math from a learned representation @math @cite_8 . It consists of two parts, an encoder @math , which encodes the input @math to a learned feature representation @math , and a decoder @math which attempts to recapture the original input by decoding the representation. For a deep convolutional AE the encoder @math and decoder @math each are modeled as deep-convolutional networks with parameters @math and @math , respectively. Hence, the training of a deep AE can be formalized as: A common choice for the reconstruction error @math is the mean-squared error (MSE): @math To reconstruct the image truthfully, the encoder has to encode the information of the input into the feature vector @math . To learn more suitable representations @math different variations of AE have been proposed @cite_5 @cite_25 :