In the field of deep learning, people often talk about factors of variation which, in my understanding (in terms of dimensionality reduction), are the latent variable directions capturing variability in the data. I also often come across the term distributed representations which are basically the features of the input data that are not mutually exclusive. Sometimes for object recognition, one may wish to extract distributed features of the object that are invariant to small variations (which are a result of combination of many factors of variation).On the other hand, in case of autoencoder, one needs to extract the factors of variation as completely as possible so as to minimize the reconstruction error.In the above two cases, factors of variation seem to have two contradicting roles, nevertheless they are both referred to as useful features for intermediate layers of a deep architecture. So what is the difference between factors of variation and distributed features? I am very confused.
What is the difference between distributed representations and factors of variation in deep learning
machine learning;neural networks