What are good parameters of an encoder?
I can not seem to grasp which specs make an encoder better than another one  


In general, in unsupervised settings, we want to learn the probability distribution of the data p(x) by some latent variables that explain the variations observed in the training set.  

The autoencoder family (Variational, Denoising, Contrastive, Sparse) try to approximate p(x) so we have a performance metric to tell us how our model is doing.  e.g. (negative log likelihood of p(x))


  lets say, unsupervised learning. For example, I am trying to teach my neural network to classify cats,   


If you use some autoencoder model to learn the distribution of cats, you could use the encoder part to further augmented with a linear classifier to discriminate between cats and other categories Therefore you have an intrinsic task (learn a good representation of the data distribution) and an extrinsic task (learn to classify cat vs not a cat). So you could do a hyper-parameter search for the model that best suits your problem by measuring its accuracy on the extrinsic task.

Side note:
GAN (Generative Adversarial Network) is a generative model, it provides some way of interacting less directly with this p(x) by drawing samples from it starting without any input. thus the situation here is different.