Title: An information theoretic approach to the autoencoder,
Abstract: We present a variation of the Autoencoder (AE) that explicitly maximizes the
mutual information between the input data and the hidden representation. The
proposed model, the InfoMax Autoencoder (IMAE), by construction is able to
learn a robust representation and good prototypes of the data. IMAE is compared
both theoretically and then computationally with the state of the art models:
the Denoising and Contractive Autoencoders in the one-hidden layer setting and
the Variational Autoencoder in the multi-layer case. Computational experiments
are performed with the MNIST and Fashion-MNIST datasets and demonstrate
particularly the strong clusterization performance of IMAE.