This paper proposes a novel method for factorising the information in the latent space of an autoencoder (AE), to improve the interpretability of the latent space and facilitate controlled generation. When trained on a dataset with labelled attributes we can produce a latent vector which separates information encoding the attributes from other characteristic information, and also disentangles the attribute information. This then allows us to manipulate each attribute of the latent representation individually without affecting others. Our method, matrix subspace projection, is simpler than the state of the art adversarial network approaches to latent space factorisation. We demonstrate the utility of the method for attribute manipulation tasks on the CelebA image dataset and the E2E text corpus.
@cite_18: We present a method for training variational autoencoders on labelled datasets which encode information corresponding to the labels in explicitly predetermined subspaces of the latent space. We motivate our model from both an information theoretic perspective as well as a adversarial game perspective. By separating labelled information into a less complicated space we allow the model to more easily disentangle representations. This provides a form of semi-supervised learning of attributes. Since these subspaces can be chosen a priori, setting them to be low-dimensional provides a form of dimensionality reduction. We demonstrate the utility of our model on attribute manipulation tasks with several image datasets.
@cite_11: Generative models, such as variational auto-encoders (VAE) and generative adversarial networks (GAN), have been immensely successful in approximating image statistics in computer vision. VAEs are useful for unsupervised feature learning, while GANs alleviate supervision by penalizing inaccurate samples using an adversarial game. In order to utilize benefits of these two approaches, we combine the VAE under an adversarial setup with auxiliary label information. We show that factorizing the latent space to separate the information needed for reconstruction (a continuous space) from the information needed for image attribute classification (a discrete space), enables the capability to edit specific attributes of an image.
@cite_2: This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.
More recent work has explicitly tackled this problem of separating the attribute information from the latent vector, using what we call an adversarial' approach @cite_2 @cite_11 @cite_18 : This is inspired by Generative Adversarial Networks (GANs); a new auxiliary network is added (like the discriminator in a GAN) which attempts to guess the label of the latent vector @math . So long as the label can be successfully guessed then there is still label (attribute) information present in @math , and the encoder is penalised via an additional loss term. In this way the encoder and new auxiliary network compete, with the encoder trying to remove label information from @math . This has the effect of minimising the mutual information between @math vector and label.